post google code post

Well nobody bothered to comment about the stuff i removed from google code apart from the one lad or lass who lamented the loss of some javafx demos.

I had comments open+moderated for a few weeks but got hit by spammers a couple of days ago so had to go back to id+moderated. Maybe something got lost in those 500 bits of snot but i don't think so. The spam was quite strange; most mentioned web sites but didn't provide links or weren't very readable so i'm not sure what the point was. Perhaps they're just fishing for open sites or naive moderators they can then exploit. Like the "windows computer department" that keeps calling and calling hoping i'll not tell them to fuck off every time (sigh, no i don't normally say that although i would tonight).

I've still got the subversion clones but i'm not inclined to do much with any of it for the forseeable future and i'm not even sure if i'm going to continue publishing other bits of code i play with going forward.

Desktop Java, OpenCL, ARM assembly language; these things are just not very common in the Free Software world. Server Java is pretty common but that's just, well, `open sauce' companies sharing costs and not hobbyists. So i think all i'm really doing is providing hints or solutions for some student's homework or help for graduate programmers to keep their jobs. And even then it's so niche it wouldn't be many, if any.

As an example of niche, I was looking up some way to communicate with adobe photoshop that doesn't involve psd format and one thing i came across was someone linking to one of my projects for some unfinished experiments with openraster format - on the first page of results. This happens rarely but still too often. Of course it could just be the search engine trying to be smart and tuning results to the user, which is a somewhat terrifying possibility (implications beyond these types searches of course). FWIW I came to the conclusion photoshop is just one of those proprietary relics from the past which intentionally refuses to support other formats so it's idiot users can continue to be arse-reamed by its inflated price.

It's just a hobby

As a hobby i have no desire to work on larger projects of my own or other established projects in my spare time. Occasionally i'll send in a patch to a project but if they want a bunch of fucking around then yeah, ... naah. In hindsight i somewhat regret how we did it on evolution but i think i've mentioned that before. Neither do i need to solicit work or build a portfolio or just gain experience.

I'm not sure how many hobbyists are around; anyone with remotely close to enough skill seems to be jumping into the wild casinos of app-stores or services and expecting to make billion$ and not just doing it for the fun of it. Some of those left over just seem to be arrogant egotistical fuckwits (and some would probably think the same of me). Same as it ever was I guess.

I suppose I will continue to code-drop even if it's just out of habit.

For another hobby I made kumquat marmalade on the weekend. Spent a couple of hours in the sun slicing the tiny fruit and extracting seeds (2-3 cups worth of seeds) and cooked it the next day. Unfortunately after all that effort it looks like it wasn't cooked quite enough and it probably wont set - it's a bit runny but at least it tastes good. Not sure what i'll do with 2-odd litres of the stuff though.

ahh shit, google code is being scrapped

Joy. I guess it's not making google enough money.

I'll make a copy of all my projects and then delete them. Probably sooner rather than later (probably immediately because i'd rather get it out of the way, i have a script going now). I'm keeping all the commit history for now although i never find it particularly useful. Links in old posts may break.

Some of them may or may not appear later somewhere else, or they may just sit on my hdd until i forget about them and delete them by accident or otherwise.

Wherever they end end up though it wont be in one of the other commercial services because i'm not interested in doing this again.

I need to do the same to this blog, ... but that can wait.

Update: So ... it seems someone did notice. Don't worry i've got a full backup of everything. In part I was pissed off with google and in part I wanted to make it immediately obvious it was going away to see if anyone actually noticed. Who could know from the years of feedback i've never received.

If you are interested in particular projects then add a comment about them anywhere on this blog. All comments are moderated so they wont appear till i let them through but there is no need to post more than once (i'll delete any nonsense or spam unless it makes you look like a bit of a wanker). I will see about enabling anonymous comments for the moment if they are not already on. I can then decide what to do depending on the interest.

I will not be using github or sourceforge nor the like.

In a follow-up post i'll see where i'm going with them as well as pondering the future location and shape of the blog itself. I've already written some stuff to take the blogger atom stuff, strip out the posts, download images, and fix the urls.

(in resp to Peter below) I knew the writing was on the wall when google code removed binary hosting: it's kinda useless for it's purpose without it. This is why all my new projects subsequent to that date are hosted differently.

on google

So google have decided to disable downloads on google code.

So I have decided to stop using it.

... although as yet I have no concrete plans or timeline for when this decision will take effect.

Whilst they claim it's about abuse, one can only assume that is just a "likely-sounding excuse" for what in reality is just another straight-up lie from the PR department of a supra-national conglomerate, and it's really just a way to cut costs and promote their 'drive' service (a useless microsoft/apple only service as far as i'm concerned).

Nobody seems to have reported that they have also gimped their POP interface to gmail a couple of days ago. No more UID support. This makes POP a lot less reliable/useful as a mail store (although in honesty it was never designed for that purpose). I proceeded to delete all the mail in gmail to help them free up some disk space.

I guess over-all the writing is on the wall. We all know that at some point 'google account' will mean 'google+', and blogger may be retired at any time.

So it seems my on-going-but-totally-lax search for alternatives to 'everything google for convenience' just got another big kick up the rump-side.

As my projects are all pretty small and low-volume I might look at a local solution because every network based solution faces the same problem. I have a couple of beagleboards doing nothing although getting a running and secure-enough system might be more pain than it's worth.

It's a bit of a pain to have to deal with.

JavaFX USB Camera App for Linux

Well I thought of something relatively simple I can do with JavaFX to get familiar with it, and maybe even end up with a useful tool. And that is a Linux web-camera application - i.e. something like Cheese.

I've been playing with the idea for a week or so but haven't felt like hacking much for various reasons. Last night was a wild storm, and the wind, rain and hail, together with the neighbours broken gutter and hayfever, it kept me up much of the night. I thought rather than struggle painfully with work i'd struggle painfully with some of my own hacking instead ...

A few hours poking around later ...

The red button takes a picture, although it just stays in memory at the moment. Then including the dynamically generated camera controls ...

Along the way I found a bug in v4l4j which was fixed very promptly - thanks Gilles.

It will let me tie in the jjmpeg code as well, if I get that far. And maybe even socles for some OpenCL processing (might get me off my arse to fix the socles filter graph stuff). Right now it doesn't do much, but on the other hand it's an almost embarassingly small amount of code too.

Layouts

So as usual - the layouts are the bane of any GUI developer. I'm using a GridPane as the base element for the on-screen controls, but it just isn't cooperating completely. e.g. the cells aren't aligning the way i tell them to - not unless I create a single-cell GridPane and align that. I guess i'm missing something, but that's all part of the learning curve no doubt.

JavaFX vs Swing

Performance is a good bit better than swing - a simple 60Hz webcam view is chugging along at about 35% cpu in Swing (the v4l4j bundled demo), whereas JavaFX is sitting around 10%. Although it goes up fairly quickly once you add blended widget overlays.

The overlays are a lot easier to code too, as are any animations and 'flashy shit', of course.

e.g. Half an hour later and I added some animation, taking a photo has it jump down to the list below, and it then fades in.

Effects

Although JavaFX has some "FX" which can be applied to scene graph nodes, i'm not sure it will be useful here. Apart from being a bit limited, the 'pulse' and asynchronous nature of the way the result can be grabbed to the host probably means it can't be used for video processing. One can have 'off-screen' scenes I believe, although i'm not sure exactly how the timing stuff is then handled.

Actually it's a task well suited to OpenCL, so if and when I get there, i'll probably just use that. It's a pity that JavaFX wont talk to OpenCL directly and the data will have to do a round-the-world trip across the PCI bus, but at webcam resolutions it shouldn't be a big issue.

As far as a 'camera app' goes, i'm personally more interested in other types of features anyway - e.g. automatic candid shooting (such as avoiding closed eyes or turned heads) for a party, or various types of image enhancement or degradation that can't be expressed using the current JavaFX effects tree.

Video ...

Another hour of hacking ... I added video recording to it via jjmpeg. I'm using mjpeg at the moment which seems to be about the only decent 'working format' I can find (fast to encode, easy to seek, commonly readable, etc). I threw everything on another thread so it easily starts recording without a glitch (at least on the workstation). It's using about 60% cpu (of 1 core) in total on this machine to capture VGA @ 60fps (although i really need to try it on a slower machine).

If only sound was that easy ...

My thinking would be that videos are recorded using a format suitable for recording (a fixed known format, high quality, low overhead, easy to manipulate), and can then be exported to another later (e.g. using ffmpeg command line).

Actually on jjmpeg, I guess I should revisit it now FFmpeg has released 1.0, maybe the api will be stable for a while. It's on the TODO list ...

I haven't checked anything in yet but it'll probably end up in mediaz/fxperiments/CamZ eventually.

Update: So I added a separate record button and flashing text and just checked it in as is, warts and all. I put it in MediaZ as a "top level" sub-project rather than inside fxperiments. Last weekend I updated jjmpeg to FFmpeg 1.0, so I changed it to use that (and found a lot of bugs along the way).

JavaFX Slideshow 2

I had another poke at the slideshow idea, I wanted to play with tiled transitions, how to do them and to see how well they performed.

Works pretty well with the GPU driver, but a little chunky as an applet or with the cpu driver - but here's the Applet & Web Start launcher of a canned demo. I just included 4 shots of mine in the jar to simplify loading them for a demo. It looks much better if it can keep up with the frame-rate requirements.

It's only doing random effects so sometimes they're a bit bland (and i'm not sure if the webstart version implements all effects), but some quite nice transitions come up sometimes. The only 'trick' is breaking the image up into multiple image views - i.e. just texture mapping squares of image separately. Each ImageView is just then animated separately after that. I had to create my own Interpolator as the default ease-in one is a bit flaccid for my liking - although creating a spline in your head is a bit hit and miss ...

Source is in MediaZ under fxperiments/FXSlideZ.

Update: For various reasons I had a lot of time and the inclination to poke today, so i hacked in a few more things into the code (this stuff isn't in the applet).

First I added a 'start from centre' translation transition for the block animator which leads to some particularly pleasant transitions. Then I played with smaller rectangles - it changes the feel of the animations more than you would think, quite pretty, but it quickly starts to bog down the transition setup code as the tiles get smaller. About 20x20 is a reasonable compromise although I left it at 40x40. At 10x10 it takes a couple of seconds to build the transition tables.

Finally I worked out 'mask' based transitions and hacked them in as a random option. I tried a couple of variations, rendering to a canvas, just animating a shape (text), and changing geometry (a filled arc). That should cover about every possibility one might want. It's mostly just down to getting the blend modes and layout right and making sure the end of the animation covers the whole window.

Smooth video list

Another morning hack: seeing if i can use JavaFX to display a smoothly operating user interface whilst crap is going on in the background.

I was thinking of something like the PlayTV library interface and so on, where one should be able to smoothly scroll even if the video is loading or something is busy. It sort of works on the PS3 but I/O really gets in the way there.

So here's a very rough prototype of the same idea - except here every video visible is also playing at the proper speed (sans audio again) as well.

Again: without seeing it in motion it doesn't really do it justice. The shot above is after scrolling a bit, and it shows the 'busy spinner' until the video starts playing, and then the video slowly fades in to cover it.

And this is a bit later. Everything is smooth and responsive and keeps up fine (on a beastly workstation of a couple years vintage). It runs ok even with the CPU driver, but the GPU one is much better.

I used a FlowPane for the layout, so it dynamically readjusts should one resize the window. I then manually calculate which parts of the FlowPane are currently visible, and use that to initiate the transition to loading the video. The video just restarts if one moves away and comes back, although if you knew the seek was going to work that would be possible too (libavformat cannot seek on some files, and the fallback is unpleasant).

I used jjmpeg for the decoding again, although this time I hit a problem with the way i was sending frames to the display - writePixels has to be on the GUI thread too. Got around it with a frame copy ... they're only small frames. It also has some pretty major memory leak somewhere which shows up when scrolling about ... might need to poke around jjmpeg some to track that or it could be errors in the listener code - i'm not really familiar with JavaFX callback rules yet.

With these particular videos (transcoded tv stuff in mp4), it still runs pretty smooth even if I maximise the window ... that's 8x7 ... 56 concurrent decodes and display (actually 64 when it's between rows). Less than 500% cpu on a 6-core HT machine (ok, so it's basically maxed out, but it still copes). (ho hum: if only the hardware video decoding wasn't locked away from being accessed by the customers who bought it, as it is, who gives a shit about that).

MediaPlayer

I also tried MediaPlayer but it wasn't so hot. I'm not sure if you can get it to scale the image data host-side, but without it it's pretty bad. Whilst it plays (oh and with sound, which isn't so good! I muted that) it's a lot slower (only some frames shown) and the whole GUI chunks around all over the place when you scroll: i.e. exactly not what I was trying to achieve in the first place. The scaling quality is poor too.

I'm assuming 'MediaPlayer' must be created from the GUI thread here, it seems to be doing a good bit of synchronous stuff when it is.

I also seem to have some memory leaks here too, but whatever the reason they add up much much faster than with the jjmpeg stuff.

Source

I checked it into MediaZ/fxperiments/VideoLibrary, and it needs a checkout of jjmpeg to work (Netbeans).

Update: I wasn't going to post it but I also created a really shitty video of it in action: didn't realise my camera only does 720P@24, and the aliasing of the LCD pixels is very poor. I also downed the bitrate to make it smaller. But you can get an idea of how it runs anyway.

• javafx-library.mp4 (there is no sound)

Simple infinite canvas

Just for a laff I coded up an 'infinite canvas' thing in JavaFX.

In the back of my mind I have the idea of making a JavaFX version of ImageZ ... but I'm also not sure I could be bothered going the whole hog. Although i could dump much of the code (although no more float support), there's still a lot of crap to deal with implementing basic features like undo.

This runs faster with the CPU renderer than with the GPU one. Drawing is about the same, but if you end up with a scene with many tiles created, the CPU pans faster ... which isn't what you'd expect. It can easily gobble up the whole heap if you create some gigantic image by zooming out a long way.

I didn't originally intend to, but since it was there I also copied the applet + webstarter to the site as well. The main reason is i've not yet had any such JavaFX applets work - but since this one did, there it is. One might need the very latest JRE for it to work (1.7.0-7). Update: I couldn't get it to work on my laptop, but that might be an old firefox - it ran fine from the jar.

I also started a new sub-project `fxperimentz' in MediaZ to chuck these experiments.

Update: Oh, i forgot to check the source in, it's there now.

PDFZ searching

So after finding and fixing the bug in my outline binding - a very stupid paste-o - I added a table of content navigator to PDFReader, and then I had a look at search.

Which ... I managed to get working, at least as a start:



As can be seen, it's a little flaky - mupdf is adding spaces here and there in the recovered text, and i'm not sure i'm processing the EOL marker properly (and possibly I have a bug in the search trie code too). But as I said - it's a start.

I decided to use a Trie for the search (Aho-Corasick algorithm) - because I know it's an efficient algorithm, and because I know there was a good implementation in evolution. So I grabbed an old copy from the GPL sources and modified it to work on the mupdf fz_text_span code. Thanks Jeff ;-) Basically it's a state machine that can match multiple (possibly overlapping) words whilst only ever advancing the search stream one character at a time.

I tried to copy the emacs mode of searching to some extent:

• / or ctrl-s starts a search
  
• The search updates immediately for the current page as characters are typed.
  
• The next search on the current page is highlighted when ctrl-s is pressed again.
  
• If there are no more results on the current page, the search starts scanning the document if ctrl-s is pressed again.
  
• ESC closes the command prompt.

I put some code in there to abort the search if the page is changed while it's still searching (because I hooked the search into the page loader/renderer), but on the documents i've tried it on it's been so fast I haven't been able to test it ...

This is one of the first times I used JNI to create complex Java objects from C - the array of results for a given page. It turns out it's fairly clean and simple to do.

I guess the next thing is to see if i can integrate the search functionality into ReaderZ. Time to write one of those horrible on-screen keyboards I guess ...

But for now ... the weather's way too nice to be inside, so I think it's off to the garden, beer in hand ...

PDFZ stuff

Amongst other hacking, I poked around with PDFZ a bit today:

• Tried to bind the outline interfaces, but either i made a big mistake or something odd is going on: the pointer I allocate in the creation function wont resolve when I go to use it (or, it resolves to NULL). Spent a long time going nowhere on that one.
  
  
• Ported some of the ReaderZ code to swing and made a simple desktop PDF viewer. It renders on the fly as you pan around, and works quite well. I use another thread to load and render pages as in ReaderZ. I'm a bit dented that I couldn't get the outline stuff to work.
  
  
• Played with an alternative page rendering method: rendering directly to the array of a BufferedImage using GetPrimitiveArrayCritical(). Can't tell the difference on the desktop but it might be useful on the kobo, and could use less memory anyway.
  
  
• Noticed that mupdf is undergoing rapid development at the moment; i'm not trying to track it at all.
  

Update: I got in touch with the mupdf devs, and I found out the next release is targeted by the end of the month; this will be a good opportunity to sync up with the api whilst the project is still fresh in my mind.

Paged layout, busy dot.

After poking around at jjmpeg a bit this morning, I played a bit more with ReaderZ. First I added an animated 'busy' icon for when the reader is busy, and moved the epub html loader to another thread so it animates. It's ugly, but it works. I simplified the use of the event manager as well.

Then I redesigned the BlockLayout code in CSZ so that I could sub-class it to create a paged media layout. It isn't 'conformant' by any stretch, and has a bug with tall images, but at least it forces lines to align to a new page once they've overflowed the viewport.

During this I realised I probably wont be able to get away with a single-pass for the layout. e.g. if you have an auto-sized box, it's size depends on things like the size of floats and the lineboxes inside of it. But you have to lay these all out before you can determine what it is, and then must lay them out again afterwards once you've determined the real size you're working with (also required for things like text-align). I might have to lay out individual words instead so then the second layout can be fast as well as letting the layout be handled separately from the text object.

e-reader, epub

Had a mini hack-fest today, and whipped up an e-pub backend for ReaderZ based on CSZ.



I added very basic img tag support as well, as is obvious.

Apart from the code to parse the content.opf file from the .epub archive which was fairly small, I spent the most time trying to work out a URL handler for a made-up 'epub:' protocol. I copied the way the jar: protocol handler distinguises between the base archive and the filename using "!/" - this is so that the normal url resolution mechanism work. But I also wanted to resolve by the manifest ID and I use the url fragment for that (although in hindsight I probably don't need it). But anyway in the end it wasn't much code, and having it there made everything 'just work', which was nice.

I also had to deal with all the crap XML brings along: i.e. dtd resolution.

The actual viewer is a bit unwieldy as it works as a set of html pages. So you need to pan around to read each 'page' (i.e. chapter, or whole book), and changing pages flips between the items in the spine (i.e. chapters or whole book). To do better than that I really need a paginating layout engine: which is something for later.

I have no svg support not surprisingly, so title pages which are pure svg come up a re-assuring blank.

Still a bit slow opening new chapters, but what can you do eh?

It's all been checked in to ReaderZ and CSZ.

floats n stuff

I made some more progress on CSZ. The latest thing I have sort-of working are floats.


I think i'm interpreting the bits i've implemented correctly: floats are quite limited so the layout logic isn't terribly complex. I still have no borders or padding (and I removed the fudge factor I had in before) so it looks a bit cramped.

It's still sad just how much crap you need to get to even this point ...

I just got a call from work and they want me back in a couple of weeks, so I might turn down the effort a bit so I can psych myself up for that. Maybe i'll finally use that kobo as a reader of books too ...

CSZ

Well I kept poking away at the XHTML/CSS stuff, for want of something better to do. I had a couple of wins along the way.

The cascading and inheritance is working somewhat better now, and I added a few more properties. Given that I'm not very familiar with all the various rules, I think I have a fairly efficient resolution mechanism by indexing various bits and pieces. The layout system is still crap, and very very incomplete, but at least I have baselines aligned now.

This is a totally contrived example, everything apart from the text layout and typefaces are hacked in one way or another.

I tried it on the kobo ... it's still fairly slow, but it's better (I think: TBH i can't remember what I tried the other stuff on). Still hampered by the text layout though.

For a 200k file (which is mostly just <p> elements), scanning the file, resolving the properties and generating the box list takes only a fraction of a second. I'm not trying to resolve or use very many properties though. About all i'm using `in anger' are some of the font-* properties.

But then performing the layout (as a single page) takes about 7s (once the jvm is warmed up), which is mostly due to TextLayout. I will have to try it with a simpler font than the one the JVM comes with.

Rendering is fairly ok (relative to the e-ink anyway) and all i'm doing is painting every textlayout in the whole tree ...

I uploaded it to MediaZ anyway, in the new CSZ module.

Hmmm, I should really take a break from hacking for a bit. But i'll believe that when I see it.

xhtml, css, boxes n shit

For some stupid reason I delved into CSS and XHTML and rendering thereof.

CSS is so deceptively simple: a few boxes, layouts in lines, and a few properties to set. Saying the devil is in the details here isn't doing the term justice. It's all in the details. And they're ugly.

From the fairly complex cascading rules, to the number of properties. The layout merging. The badly written documentation: filled with "x inline y box" "a block b box c" to such specificity, and no with definitions it is quite difficult to decipher what it's even talking about. It's also quite hard to debug, since it needs a fair bit of data structure to represent it.

CSZ

Anyway, after some mucking about, I have a relatively complete CSS lexer and parser, a fairly incomplete cascade resolver, a fairly incomplete layout engine, and a very incomplete style system. It's just enough to show paragraphs of text with some basic formatting. For a book reader I don't want the document to control the text too much anyway.

I'm attempting to do it while streaming the input, and (obviously thus) in a single pass using the pull parser from XMLStreamReader. Therefore initial parsing is quite quick, but it's still taking a relatively long time to lay out the boxes ...

TextLayout

And the problem here is TextLayout. It's just quite slow. I tried my own version of layout using FontMetrics.getCharsWidth(), but inside that just creates a TextLayout anyway, so it's even slower (or maybe not, now it's a bit faster?).

I know why it's so complicated; for laying out complex scripts and handling all the special cases. Anyway, that is the primary factor of constraint on performance at the moment, although as the implementation is so far from finished, i'm sure it wont be the last one.

(I played some more, and the font used plays a big part in the speed taken, so there's hope yet).

I suppose I should try it on the kobo to see how it goes there.

Dead end?

It's taken a lot of effort to get this far, and i'm not really happy with the result. So i'm not sure if i'll keep plugging away at it or throw it away (and if i need such a functionality, use cssbox). There's a lot left to get it to be useful for anything.

ReaderZ

I finally got around to checking in ReaderZ to mediaz.

I also tweaked a few things before I did:

• Added timer and timeout functionality to the GagdetZ event thread. Using a ScheduledExecutionService made this an absolute doddle, and infact the 'event thread' is now just the service.
  
  
• Changed the update mechanisms to try to speed up panning. It now pans using black and white updates and then does a full 'invert' refresh once you stop - after a 700ms timeout of inactivity. It is still a bit slow updating, but that might be because i'm rendering the content on the fly. I can't seem to get the e-ink to go any faster, although I did speed that up a bit over what it was.
  
  
• I took out the HTML backend for now. I'm using a very slightly modified build of CSSBox (to remove the logging dependencies) and just can't be bothered to work that all out. It's a bit slow anyway.
  
  
• For fun I added a picture viewer backend to the DocumentReader.
  
  
  

The README has all the other gory details.

I also had a good look at the text layout mechanism in Java ... boy what a nightmare of code that is. No wonder it's so slow. I tried to work out how it was doing it when it came down to it, but I couldn't fathom it before losing interest. The kobo html reader is about 10x faster at pagination; which just makes it all the more puzzling as to why the text reader is so slow.

Which has me thinking about C again, and using mupdf's stuff to format and render text instead ... well it would work for latin scripts anyway.

A browser ... ?

Ahh, so I totally didn't think I was going to even try to do this ...

But epub needs HTML, and I found CSSBox, and well one thing lead to another ...


So I've basically ported the SimpleBrowser example from CSSBox to ReaderZ - all it can do is render the page, and it lets you pan and zoom as with PDF files. No links or anything.

I tried rendering on the fly, and into an image - the former is a little slow to scroll (but not far off the e-ink speed), but I don't think it's clipping the drawing regions properly and so doing a lot more work than necessary anyway. Using an image scrolls fast but can't be zoomed very well, and takes more memory (I blew it with boycottnovell) ... so trade-offs.

It's pretty slow and clunky, but what can one expect when XML is involved?

Reader Shell

So I kept poking away at the browser code and my gadget toolkit.

It's getting fairly sophisticated now: I improved the StackLayout gadget to add filling and relative sizing glue. I added a list gadget - which works by pages, and is similar to JList, including a cell renderer, list model and selection model and I played around with a whole lot of other stuff as well.

So I have enough to finally create a reader shell: it presents a list of the files on the device, and lets you open them up with a pdf reader or a text reader, depending on the file type. Once inside it has a full-screen viewer with no visible buttons - but without buttons you can zoom, pan, change pages one at a time or flip through many a time. A popup menu (single short press in the middle of the screen) lets you quit back to the shell.

I cache the pagination for text files, so they open very quickly after the first visit, assuming the font settings haven't changed. The pagination descriptor is small, about 7k for a 500 page document. On a re-visit of the same file (i.e. once the jvm is warmed up), it's loading 500k text file in about 1/2 a second but even from a cold-start it's only about a second. PDF files also open fast, about the same speed. i.e. not much slower than the e-ink can refresh a single greyscale page. Closing a file and returning to the shell is similarly speedy.

And whilst the interface isn't very complicated, everything is still quite responsive, with no missed finger presses or long unexplained delays (although the first time you open a 500k text file, it still takes about 25s to re-paginate).

My panning is still a bit slow, although I am rendering the bitmap on the fly as well.

But ... i've pretty much done what I want for now: prove that the kobo touch e-reader is a zippy little unit, capable of much more performance than the included software lets it deliver.

I'll drop the code in MediaZ sometime in the next few days, and continue working on it for at least a while.

PDFZ

Checked in the first cut of PDFZ to the MediaZ project.

It's a simple binding to muPDF - i.e. a PDF renderer for Java that builds on arm and x64/x86 cpus.

Touch Interfaces 2

I was wondering how to get a decent user interface with only a finger ...

I came up with the idea of sensors (no doubt this is what everyone else already does, but I haven't used a touch device before let alone coded for one) that I can attach to a gadget, and the gadget can then decide what to do.

I have two types of sensors, drag sensors and press sensors. Drag sensors override any press sensors, and come into action once a drag has started. It only started after a certain threshold of movement is exceeded. Press sensors only activate if a press is detected without a drag. Additionally they can handle long (>300ms) or short presses separately.

So for a PDF viewer, I came up with the following 'soccer pitch' of sensors ...

• The semi-circle sensors are press sensors, and are used to pan around the page in screen-sized chunks in the obvious directions. I haven't implemented these yet.
  
• The left and right thirds of the entire screen are for page turning.
  
• The thin rectangle at the top of the screen is a drag sensor used for changing the zoom. There is also a coincident press sensor which is used to reset the zoom.
  
• The thin rectangle at the bottom of the screen is a drag sensor used for flipping through pages.
  
• The whole screen itself is a drag sensor, and used for finger panning. The whole screen also has a press sensor which will cause an e-ink 'cleanup' on a long press.

As far as this goes and even without implementing the panning buttons it makes a fairly comfortable PDF reader out of such a small screen. I might also need a way to go back to the top of the page but one-screen over, and back to the left of the page but one-screen down (and backwards of those?); although there is only so much space for such hidden buttons to be easy to use. I could either use the corners, or perhaps just do it based on exceeding the limit of the existing paging buttons.

PDFZ on Kobo

Well I got the PDF reader working on the kobo. The binding library 'just worked' thankfully, so I just cleaned the code up and worked on some performance issues.


Using a custom BufferedImage which talks directly to the ByteBuffer turned out to be very slow on the kobo jvm - about 10x slower than just copying the ByteBuffer to an RGB565 BufferedImage in a manual Java loop. So I just do that ... the images are only the size of the display so memory isn't a big issue.

It loads and renders a page in about 0.3s, so that is fast enough. I couldn't work out how to get it to 'fully' refresh automatically very cleanly so I just added a button to do it. Panning is a bit slow, even with monochrome update mode: OTOH finger events aren't lost when you press-slide-quickly-release.

For comparison here's the same text with the built-in viewer.


I think my last mention if it was a bit unkind - it isn't that bad, it turns out I had been reading a scanned paper, which tends to render poorly in everything. I found out that you can change pages whilst zoomed, although you either have to scroll off the side of the image first, or bring up the menu - and while the menu is up you get an icon of the 'zoom area' taking up a good chunk of the screen, so they're both a little bit clumsier than they might be. And some contrast enhancement wouldn't go astray when viewing text (although IMHO mupdf just renders text very well).

Incidentally, the dark square you can see in the lower left is really there: it's left over from an e-ink refresh after the zoom-preview-box is removed.