This will be my last post to a-hackers-craic!
I've moved all the posts and images to http://www.zedzone.space/blog, although how they look and how they work are still very much work in progress. I had already moved most of http://users.on.net/notzed to http://www.zedzone.space yesterday.
Some things might be broken for a while but everything apart from comments should have already made it across intact.
Byeee.
So I guess the mood took me, I somehow ended poking away until the very late morning hours (4am) the last couple of nights hacking on jjmpeg. Just one more small problem to solve ... that never ended. Today I should've been working but i've given up and will write it off, it's nearly xmas break anyway so there's no rush, and i'm ahead of the curve anyway.
I got this ported over and playing video fairly easily, and then went through on a cleanup spree. I removed all the BufferedImage, multi-buffering, and scaling stuff and a few other experiments which never worked. Some api changes allowed me to consolidate more code into a base class, and some changes to AVStream necessitated a different approach to initialising the AVCodecContext (using AVCodecParameters). I made a few other little tweaks on the way.
The reason I removed the BufferedImage code is because I didn't want to pollute it with "platform specific" code. i.e. swing, javafx, etc. I've moved that functionality into a separate namespace (module?).
My first cut just took the BufferedImage code and put it into another class which provides the functionality by taking the current AVFrame from the JJMediaReader video stream. This'll probably do but when working on similar functionality for JavaFX I took a completely different approach - implementing a native PixelReader() so that the native code can decide the best way to write to the buffer. This is perhaps a little more work but is a lot cleaner to use.
jjmpeg1 lets you scale images 'directly' to/from primitive arrays or direct ByteBuffers in addition to AVFrame. Since they have no structure description (size, format), this either has to be passed in to the functions (messy) or stored in the object (also messy). jjmpeg1 used the latter option and for now I simply haven't implemented them.
The PixelReader mentioned above does implement it internally but for code re-use it might make sense to implement them with the structure information as explicit parameters, and use higher level objects such as PixelReader/Writer to track such information. On the other hand the native code has access to more information so it also makes sense to leave it there.
I went a bit further and created a re-usable super-class that does most of the work and toolkit specific routines only have to tweak the invocation. This approach hides libswscale behind another api. The slice conversions don't work properly but they're not necessary.
So far I had public constructors and `finalisers' because otherwise the reflection code failed. That's a bit too ugly (and `dangerous') so I made them private. The reflection code just had to look up the methods and set them Accessible.
Constructor cc = jtype.getDeclaredConstructor(Long.TYPE);
cc.setAccessible(true);
return cc.newInstance(p);
Whilst working on JJMediaReader I hit a snag with the issue of ownership. In most cases objects are either created anew and released (or gc'd) by the Java code, or are simply references to data managed elsewhere. I was addressing the latter problem by simply having an empty release() method for the instance, but that isn't flexible enough because some objects are created or referenced the the context determines which.
So I expanded the Java-side object tracking to include a `refer' method in addition to the 'resolve' method. `resolve' either creates a new instance or returns and existing one with a weak-reference object which will invoke the static release method when it gets finalised. `refer' on the other hand does the same thing but uses a different weak-reference object which does nothing.
I then noticed (the rather obvious) that if an object is created, it can't possibly 'go away' from the object tracking if it is still alive; therefore the `resolve' method was doing redundant work. So I created another `create' method which assumes the object is always a new one and simply adds it to the table. It can also do some checking but i'm pretty sure it can't fail ...
If on the other hand the underlying data was reference counted then the `resolve' method would be useful since it would be possible to lookup an existing object despite it being `released'. So i'll keep it in CObject.
As part of this change I also improved CObject in other ways.
I was storing the weak reference to the object itself inside the object so I could implement explicit release and to avoid copying the pointer. I removed that reference and only store the pointer now. The WeakReference it already tracked in a hash table so I just look it up if I need it. This lets me change the jni code to use a field lookup rather than a function call to retrieve it (I doubt it makes much perf difference but I will profile it at some point).
I also had some pretty messy "cross-layer" use of static variables and messy synchronisation code. I moved all map references to outside of the weak reference routine and use a synchronised map for the pointer to object table.
For explicit release I simply call .clear() and .enqueue() on the WeakReference - which seems to do the right thing, and simplifies the release code (at least conceptually) since it always runs on the same thread.
This also supports `safe' explicit release which may be required in some circumstances where the gc is not run often enough.
It should work well with the JVM as it uses reference queues and no finalize methods. It requires minimal "extra" application support - just a class specific release() method.
The following few lines is now the complete makefile for dez. This supports `jar' (normal build target), `sources' (ide source jar), `javadoc' (ide javadoc jar), dist (complete rebuildable source), and now even `test' or `check' (unit and integration tests via JUnit 4) targets. The stuff included from java.make is reusable and is under 200 lines once you exclude voluminous comments and documentation.
java_PROGRAMS = dez dez_VERSION=-1 dez_JAVA_SOURCES_DIRS=src dez_TEST_JAVA_SOURCES_DIRS=test DIST_NAME=dez DIST_VERSION=-1.3 DIST_EXTRA=COPYING.AGPL3 README Makefile include java.make
The article is over on my home page at Using GNU Make for java under my software articles section.
This morning I wrote and published article about writing an image container class for Java which supports efficient use of Streams. It is on my local home page under Pixels - Java Images, Streams.
Although there is much said of it, there is still quite a bit unsaid about how many wrong-footed experiments it took to accomplish the seemingly obvious final result. The code itself is now (or will be) part of an unpublished library I apparently started writing just over 12 months ago for reasons I can no longer recall. It doesn't have enough guts to make publishing it worthwhile as yet.
I'm also still playing with fft code and toying with some human-computer-interaction ideas.
After posting the result I kept experimenting with the code I `live blogged' about yesterday. I did some linlining but was primarily experimenting with multi-threading. I also looked at a decimation in time algorithm (which i kept fucking up until I got it working today), and a coupe of other things too, as will become apparent.
First a picture of a thousand words.
Now the words.
This picture shows the CPU load over time (i'm sure you all know what that is) as I ran a specific set of tests on a test against 8x runs of a 2^24 complex forward transform. The lines represent each core available on this computer. I put some 2s sleep calls between steps to make them distinguishable. Additionally each horizontal pixel represents 250ms which is is about the minimum sampling time which gives usable results.
Refer to earlier posts as to what the names mean.
That's the setup out of the way, the first 3rd or so of the plot. It's important but not as important as the next bit.
Well that's it I suppose. It utilises more of the available cpu resource and executes in a shorter time. But how was that achieved?
As Deane would say, ``Well Rob, i'm glad you asked''.
It only required a couple of quite simple steps. Firstly I copied the "radix4" routine into the inner loop of "radix4_pass". The jvm compiler wont do this itself without some options and it makes quite a difference of itself. Then I copied this to another "radix6_pass" which takes additional arguments that defines a sub-set of a full transform to calculate. I then just invoke this in parts from 4x separate threads, and keep doing that sort of thing until I hit the "logSplit" point and subsequently proceed as before. It was a quick-and-dirty and could be cleaned up but probably wont add much performance.
It was a bit of mucking about with the addressing logic but once done it's actually a fairly minor change: yet it results in the best performance by far.
At this point i've explored all the isssues and am working on a complete implementation which ties it all together. I think I will write two implementations: one using fully expanded tables for "ultimate performance" and another which calculates the Wn exponents on the fly for "ultimate size". Today I got a DIT algorithm working so I will fill out the API with forward/inverse, pairs-of-real, real-input, perhaps in-order results, and a couple of other useful things to aid convolution performance. Oh and 2D of all that.
But now for a little rant.
So as part of this effort, I ended up having to write my own cpu load monitor. The only one I had available on slackware only has a tiny graph and is mostly just a GUI version of top. The dark slate blue is user time, the grey area is idle, crimson is cpu load, irq is medium sea green and the io wait is golden rod. The kernel doesn't report particularly accurate values in /proc/stat but it sufficed.
But I had intended to annotate this image with some nice 'callouts' and shadowed boxes and whatnot so i wouldn't have to write those 1000 words just to explain what it was showing. However ...
gimp has turned into a "professional photographer editing suite" - i.e. a totally useless piece of junk for most of the planet (and pro photographers wont use it anyway?). So the only other application i had handy was openoffice "dot org" (pretentious twats) draw. I even started to track down the dependencies of inkscape to build that but gtkmm? Yeah ok. But openoffce: Jesus H fucking-A-cunt-of-a-christ, what a load of shit that is. I can't imagine how may millions of dollars that piece of rubbish has cost in terms of developer hours and wasted customer time (aka luser `productivity') but i'm astounded by just how terrible it is. It runs very slow. Has some weird-arse modality/GUI update bullshit going on. Is a total pain to use (in terms of number of mouse clicks required to do the most trivial of operations). And above that it's just buggy as hell. I'd call up the voluminous settings "dialogue" to change a background colour and then it would decide to throw any changes i'd made if i didn't explicitly set it every time.
But it's in good company and about as shit as any `office' software has ever been since the inane marketroid idea to lock users into fucktastic `software ecosystems' was first conceived of. Fuck micro$oft and the fucking hor$e it fucking rode in on.
I was so pissed off I spent the next 4+ hours (till 5am) working on my own structured graphical editor. Ok maybe that's a bit manic and it'll probably go about as far as the last 4 times I did the same thing the last 4 times I also tried using a bit of similar software to accomplish a similar seemingly-simple goal ... but ``like seriously''?
To phrase it in the parlance of our time: what in the actual fuck?
Just for something a bit different this morning I had an idea to do a record of developing software from the point of view of a "live blog". I was somewhat inspired by a recent video I saw of Media Molecules where they were editing shader routines for their outstandingly impressive new game "Dreams" on a live video stream.
Obviously I didn't quite do that but I did have a hypothesis to test and ended up with a working implementation to test that hypothesis, and recorded the details of the ups and downs as I went.
Did I get a positive or negative answer to my question?
To get the answer to that question and to get an insight into the daily life of one cranky developer, go have a read yourself.
I started writing a decent post with some detail but i'll just post this plot for now.
Ok, some explanation.
I tried to scale the performance by dividing the execution time per transform by N log2 N. I then normalised using a fudge factor so the fastest N=16 is about 1.0.
My first attempt just plotted the relative ratio to jtransforms but that wasn't very useful. It did look a whole lot better though because I used gnuplot, but this time i was lazy and just used openoffice. Pretty terrible tool though, it feels as clumsy as using microsoft junk on microsoft windows 95. Although I had enough pain getting good output from gnuplot via postscript too, but nothing a few calls to netpbm couldn't fix (albeit with it's completely pointless and useless "manual" pages which just redirect you to a web page).
Well, some more on the actual information in the picture:
int l1 = i << logStep;
int l2 = i << logStep + 1;
int l3 = l1 + l2;
int mw = w.length - 2;
int nw = w.length - 1;
float w1r = w[l1];
float w1i = -w[nw - l1];
float w2ra = w[l2 & mw];
float w2ia = w[nw - (l2 & mw)];
int cs2 = (l2 >> logN - 1) * 4;
int cs3 = (l3 >> logN - 1) * 4;
float w2r = cosmap[cs2 + 0] * w2ra + cosmap[cs2 + 1] * w2ia;
float w2i = cosmap[cs2 + 2] * w2ra + cosmap[cs2 + 3] * w2ia;
float w3ra = w[l3 & mw];
float w3ia = w[nw - (l3 & mw)];
float w3r = cosmap[cs3 + 0] * w3ra + cosmap[cs3 + 1] * w3ia;
float w3i = cosmap[cs3 + 2] * w3ra + cosmap[cs3 + 3] * w3ia;
And this.
So it turned out and turns out that the twiddle factors are the primary performance problem and not the data cache. At least up to N=2^20. I should have known this as this was what ffts was addressing (if i recall correctly).
Whilst a single table allows for quick lookup "on paper", in reality it quickly becomes a wildly sparse lookup which murders the data cache. Even attempting to reduce its size has little benefit and too much cost; however 'tab_1' does beat 'tab_0' at the end. While fully pre-calculating the tables looks rather poor "on paper" in practice it leads to the fastest implementation and although it uses more memory it's only about twice a simple table, and around the same size as the data it is processing.
In contrast, the semi-recursive implementation only have a relatively weak bearing on the execution time. This could be due to poor tuning of course.
The rotation implementation adds an extra 18 flops to a calculation of 34 but only has a modest impact on performance so it is presumably offset by a combination of reduced address arithmetic, fewer loads, and otherwise unused flop cycles.
The code is surprisingly simple, I think? There is one very ugly routine for the 2nd to lass pass but even that is merely mandrualic-inlining and not complicated.
Well that's forward, I suppose I have to do inverse now. It's mostly just the same in reverse so the same architecture should work. I already wrote a bunch of DIT code anyway.
And i have some 2D stuff. It runs quite a bit faster than 1D for the same number of numbers (all else being equal) - in contrast to jtransforms. It's not a small amount either, it's like 30% faster. I even tried using it to implement a 1D transform - actually got it working - but even with the same memory access pattern as the 2D code it wasn't as fast as the 1D. Big bummer for a lot of effort.
It was those bloody twiddle factors again.
Update: I just realised that i made a bit of a mistake with the way i've encoded the tables for 'tab0' which has propagated from my first early attempts at writing an fft routine.
Because i started with a simple direct sine+cosine table I just appended extra items to cover the required range when i moved from radix-2 to radix-4. But all this has meant is i have a table which is 3x longer than it needs to be for W^1 and that W^2 and W^3 are sparsely located through it. So apart from adding complexity to the address calculation it leads to poor locality of reference in the inner loop.
It still drops off quite a bit after 2^16 though to just under jtransforms at 2^20.
Well I did some more mucking about with the fft code. Here's some quick results for the radix-4 code.
First the runtime per-transform, in microseconds. I ran approximately 1s worth of a single transfer in a tight loop and took the last of 3 runs. All algorithms executed sequentially in the same run.
16 64 256 1024 4096 16384 65536 262144
jtransforms 0.156 1.146 6.058 27.832 135.844 511.098 3 037.140 14 802.328
dif4 0.160 0.980 5.632 27.503 138.077 681.006 3 759.005 20 044.719
dif4b 0.136 0.797 4.713 22.994 120.915 615.623 3 175.115 17 875.563
dif4b_col 0.143 0.797 4.454 21.835 117.659 593.314 3 020.144 22 341.453
dif4c 0.087 0.675 4.255 21.720 115.550 576.798 2 775.360 15 248.578
dif4bc 0.083 0.616 3.760 19.596 108.028 547.334 2 810.118 16 308.047
dif4bc_col 0.137 0.622 3.699 19.629 107.954 550.483 2 820.234 16 323.797
And the same information presented as a percentage of jtransforms' execution time with my best implementation highlighted.
16 64 256 1024 4096 16384 65536 262144
jtransforms 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0
dif4 102.4 85.5 93.0 98.8 101.6 133.2 123.8 135.4
dif4b 86.7 69.6 77.8 82.6 89.0 120.5 104.5 120.8
dif4b_col 91.3 69.6 73.5 78.5 86.6 116.1 99.4 150.9
dif4c 55.9 58.9 70.2 78.0 85.1 112.9 91.4 103.0
dif4bc 53.3 53.8 62.1 70.4 79.5 107.1 92.5 110.2
dif4bc_col 87.7 54.3 61.1 70.5 79.5 107.7 92.9 110.3
Executed with the default java options.
$ java -version java version "1.8.0_92" Java(TM) SE Runtime Environment (build 1.8.0_92-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)
CPU is kaveri clocked at minimum (1.7Ghz) with only a single DIMM, it is quite slow as you can see.
A summary of the algorithms follow.
The final (trivial) pass is hand-coded in all cases. dif4 requireds N/2 complex twiddle factors and the rest require N/2+N/4 due to the factorisation used.
In all cases only a forward complex transform which leaves the results unordered (bit-reversed indexed) is implemented.
All the implementations do full scans at each pass and so start to slow down above 2^12 elements due to cache thrashing (together with the twiddle table). Depth-first ordering should help in most cases.
Despite requiring fewer twiddle lookups in dif4c the extra complexity of the code leads to register spills and slower execution. That it takes a lead at very large numbers is also consistent with this and the point above.
Twiddle factor lookups are costly in general but become relatively costlier at larger sizes. This needs to be addressed separately to the general cache problem.
Whilst the re-ordering for the "col" variants was a cheap and very simple trick to gain some performance, it can't keep up with the hand-coded variants. Further tuning may help.
For addressing calculations sometimes it's better to fully spell out the calculation each time and let the compiler optimise it, but sometimes it isn't.
It is almost always a good idea to keep loops as simple as possible. I think this is one reason such a trivial / direct implementation of the FFT beats out a more sophisticated one.
There's a pretty clear winner here!
In most cases the calculations are expanded in full inside inner loops, apart from the W^0_N case. In all other cases the compiler generated slower execution if it was modularised, sometimes significantly slow (10%+). I suspect that even the radix-4 kernel is starting to exhaust available registers and scheduling slots together with the java memory model. The code also includes numerous possibly questionable and sometimes a little baffling micro-optimisations. I haven't validated the results yet apart from a test case which "looks wrong" when things go awry but i have reasonable confidence it is functioning correctly.
After I found a decent reference I did start on a radix-8 kernel but i realised why there is so little about writing software to do it; it just doesn't fit on the cpu in a high level language. Even the radix-4 kernel seems to be a very tight fit for java on this cpu.
Still playing ...
After quite a break i finally got stuck into a bit of hacking over the weekend. Actually i literally spent every waking hour of the weekend tinkering on the keyboard, about 25 hours or so. Yes my arse is a bit sore.
I will probably end up writing more about it but for now I'm just going to summarise some findings and results of some tinkering with FFT routines in Java. It may sound oxymoronic but i've been a bit under the weather so this was something "easy" to play with which doesn't require too much hard thought.
Initially i was working on an out-of-place in-order implementation but I hit some problems with 2D transforms so decided to settle on in-place out-of-order instead. Using a decimation in frequency (DIF) for the forward transform and a decimation in time (DIT) for the inverse allows the same routine to be used for all passes with no bit-reversal needed. I only implemented the basic Cooley-Tukey algorithm and only with radix-2 or radix-4 kernels and with special case code for the first (DIT) or last (DIF) pass. I'm only really interested in "video-image-sized" routines at this point.
I normally use jtransforms (2.4) for (java) fft's so i used that as a performance basis. Whilst the out-of-order results aren't identical they are just as useful for my needs so I think it's fair to compare them. In all cases i'm talking about single-threaded performance and complex input/output (jtransforms.complexForward/complexInverse), and i've only looked at single precision so far.
An all radix-2 implementation is about half the speed of jtransforms but takes very little code.
A naive radix-4 factorisation (40 flops) is roughly on-par with jtransforms depending on N.
A better radix-4 factoriation (34 flops) is 10-30% faster across many sizes of interest.
I can beat jtransforms quite handily at 2D transforms even using the naive radix-4 kernel. e.g. ~60% for 102x1024.
This only required creating a somewhat trivial implementation of a column-specific transform. More on this in another post. It will have some pretty pictures. I'm still working on code for it too.
The performance inflexion point (1D) is about 2^14 elements, beyond that jtransforms pulls (well) ahead. I suspect this is related to the cache size and such minutiae is something i just haven't explored.
Some other java/jvm related observations:
Single micro-optimisations can make significant (up to 20%) differences to total runtime. Such as:
Memory access ordering is particularly difficult to overcome; there is no 'restrict' keyword or even possibility with the java language. Any changes one makes will be quite platform and compiler specific (although microoptimisations in general are to start with). Nope! Seems i was just a victim of a skinner box. After a check I found there are no relevant rules here and changes i measured were for unrelated internal compiler reasons which are of no practical use. Should've checked before wasting my time.
Floating point arithmetic is also a bit-shit-by-design so expressions can't be freely re-arranged by the compiler for performance (well, depending on how reliable you expect the results to be, i think java aims to be strict). Java currently also has no fused-multiply-add although java-9 has added one. "complex float" would be nice. Although I didn't feel like digging out the disassembler to explore the generated code from hotspot so i'm only guessing as to how it's compiled the code.
I suspect it inlines too much and causes register spills. In some cases I saw 20%+ performance hits although after upgrading to the latest jvm (from u40 to u92) they seem less severe although still present. I used "-XX:CompileCommand=dontinline,*.*" to compare to the default. It showed up as the first-of-3 runs being somewhat faster than subsequent ones, whereas normally it should improve as hotspots are identified. I just thought it was my cpu throttling at first but it continued after i'd locked it.
On a shithouse register starved architecture like x86 the jvm overheads limit the complexity of routines. e.g. a (naive) radix-8 kernel ran slower than a radix-4 one due to register spills (it wasn't very good either though).
It's very easy to take a "simple" routine and blow it out into a big library of special cases and experiments for that elusive extra single-digit% in performance gained. Actually i've done a lot of this work before but i didn't bother referencing much of it because there is no goal; only a journey.
Of course 'performance java' isn't exactly a hot topic anymore and i should be focusing on massively-parallel devices but I was only doing it to pass the time and I can tackle OpenCL another wet weekend. If I can identify isolated cases for the the optimiser-degredations I came across I will see what the java forums have to say about them.
I've nearly exhausted my interest for now but i still have a couple more things to try and also to identify what if any of it I want to keep. If I get really keen I will see about some sort of reusable library for special purpose fft use - one problem with jtransforms (and indeed other fft libraries) is that you need to marshal the data into/from the correct layout which can be quite costly and negate much of the performance they purport to provide. Big if on that though.
Oh, and what a nice birthday present too!
Update: Not that i've had time to look but still no AMD drivers for linux; although if this means an end to fglrx then that will be worth it.
I was playing with linux signals the need arose for a lockless queue, although given the mess of poll vs signal and so on i might just use a pipe for signalling as i've done before. But such a queue is useful for that case as well.
Unless you're dealing with exotic hardware or performance I imagine that a basic bounded lockless queue is fairly easy to implement using a two-index compare and swap but I was thinking about the unbounded case. Not that unbounded is necessarily a good idea in many cases for other reasons.
After a couple of mistakes I think I arrived at a very simple solution which should work. It's actually half a stack, and half a something else I don't even know the name of, but it can be made to appear as a queue cheaply.
This is implemented as a push onto the head of a single-linked list. This can be implemented very simply using a compare and swap against the single pointer of the stack head. The next pointer of the node can be freely updated until it is owned by the queue once the compare and swap succeeds.
void enqueue(queue q, msg m) {
queue old, new;
do {
old = q;
new.head = m;
m.next = old.head;
} while (!compare_and_swap(q, old, new));
}
This is no dequeue operation, instead there is a dequeue-all which removes every element currently on the queue but returns them in reverse order. It just clears the stack head and returns it's old value. This can be implemented trivially using an atomic exchange operation with NULL.
queue dequeue_all(queue q) {
q = exchange(q, null);
return q ? q.head : null;
}
This has to be performed as a separate step but it is performed local to the reader. It should be efficient enough on typical processors. In the (very) unlikely event that order isn't important then it isn't even necessary.
void dequeue_foreach_rec(msg stack, func cb) {
if (q) {
dequeue_foreach_rec(stack.next, cb);
cb.apply(stack);
}
}
void dequeue_foreach(queue a, func cb) {
dequeue_foreach_rec(dequeue_all(q), cb);
}
It supports multiple writers. Whilst it still functions "correctly" for multi-reader queues it probably isn't terribly useful for that case as it's first-come-first-gets-all with no attempt or possibility of balanced delivery.
I added resizing. It meant I had to add some locking to the snapshot routine but it only needs to lock around the resize operation so it adds almost no overhead to normal operation (merely detecting a resize has occurred). I'm not yet sure what's supposed to happen with saved cursors and alternate screens when a resize occurs, probably just clip to size. Unfortunately there seems to be no way to set the WM_NORMAL_HINTS on the javafx stage, so there's no way to make it size to cells properly.
I did a bunch of benchmarking and profiling. One thing I tried was another test to compare to xterm - now at full-screen. Running "find . -name '*.c' | xargs cat" from the root of the linux 3.19.8 source tree. After a couple of runs this is about 25 seconds on termz, and 16 minutes in xterm. Well. Yeah it's a silly test but all those ls -l's during the day add up.
Looking at the memory profiler it doesn't really use too much heap during operation, just a few MB and most of that is the image and javfx. I mean nor should it, there isn't data structures to maintain. But having multiple compilers in ram (jvm, OpenCL), their generated output, and all the added overhead of the runtime support needed for those really adds up so it's very very fat in practice. I guess if multiple terms ran on the same jvm it would be ok.
So at this point i'm not really sure what i'll do with it. I'll probably poke at the edges when i'm bored and eventually when I get around to it I will dump what I have to my software site as the result of a weekend-and-a-bit-hack.
After that I'm not sure. It's actually quite functional and robust already (well, compared to the effort in) and wouldn't take much more work to turn it into a usable terminal for me - add some scrollback (pretty simple), mouse selection stuff (not that hard), and sort out some of the keyboard details (reading obtuse documents and testing). So maybe that will happen.
If I got that far adding the "10x20" typeface would probably be on the cards. Fixed-size outline fonts would be possible by just pre-rendering them but to me they just aren't terminal fonts.
Anything further such as a re-usable term component (which might actually be of use to the world) would require substantially more work on the i18n side of things and I don't feel like learning enough to do that properly.
I did a bit poking at the java makefile stuff and it's to the point where i'm using it for out-of-ide builds of termz and zcl and will look into using it on other projects. That's the best way to find bugs/what works and what doesn't and previous attempts never got that far. For all it's gnumakefile obtuseness it's really rather compact at under 200 lines excluding comments and I didn't put any effort into making it particularly small. And that includes targets for javadocs, source jars, and binary builds.
Today I had another go at looking at using meta-rules with gnu make to create a 'automatic makefile' system for building java applications. Again I just "had a look" and now the day is all gone.
This is the current input for termz.
java_PROGRAMS=termz termz_VERSION=0.0 termz_DISTADD=Makefile java.make \ jni/Makefile \ jni/termz-jni.c # compiling termz_SOURCES_DIRS=src termz_RESOURCES_DIRS=src termz_LIBADD=../zcl/dist/zcl.jar # packaging (runtime) termz_RT_MAIN=au.notzed.termz.TermZ termz_RT_LIBADD=../zcl/dist/zcl.jar # native targets termz_PLATFORMS=gnu-amd64 # native libs, internal or external. path:libname termz_RT_JNIADD=../zcl/jni/bin:zcl jni/bin:termz # Manually hook in the jni build jni/bin/gnu-amd64/libtermz.so: build/termz_built make -C jni TARGET=gnu-amd64 clean:: make -C jni clean include java.make
This builds the class files in build/termz, then the jni libraries via the manual hook. In another staging area (build/termz_dist) it merges the classes and the resource files (stripping the leading paths properly). It uses javapackager to create an executable jar file from this staged tree which includes references to the RT_LIBADD libs (moved to lib/). Finally it copies the RT_LIBADD jar files into bin/lib/ and the native libraries into bin/lib/{platform} so that the jar can be executed directly - only java.library.path must be set.
Not shown but an alternative target type is to use java_JARS instead of java_PROGRAMS. In this case jar is used to package up the class and resource files and in this case no staging is needed.
This tars up all the sources, resources, and DISTADD into a tar file. No staging is required and all files are taken 'in-place'. The tar file extracts to "termz-0.0/".
Blows away build and bin and (indirectly) jni/bin. All 'noise' is stored in these directories.
I'm using metaprogramming so that each base makefile can define multiple targets of different types. This is all using straight gnu make - there is no preprocessing or other tools required.
java.make does all the "magic" of course. While it's fairly straightforward ... it can also be a little obtuse and hairy at times. But the biggest difficulty is just deciding what to implement and the conventions to use for each of them. Even the variable names themselves.
There were a couple of messier problems that needed solving although i'd solved the former one last time I looked at this some time ago.
For makefile operation the filenames need to be specified absolutely or relative to the Makefile itself. But for other operations such as building source jars one needs the class-relative name. The easiest approach is just to hard-code this to "src/" but I decided I wanted to add more flexibility than this allows.
The way I solved this problem was to have a separate variable which defines the possible roots of any sources or resources. Depending on what sort of representation I need I can then match and manipulate on these roots to form the various outputs. The only names specified by the user are the filenames themselves.
For example when forming a jar file in-place I need to be able to convert a resource name such as "src/au/notzed/terms/fonts/misc-fixed-semicondensed-6x13.png" into the sequence for calling jar as "-C" "src" au/notzed/terms/fonts/..." so that it appears in the correct location in the jar file.
I use this macro:
# Call with $1=root list, $2=file list
define JAR_deroot=
$$(foreach root,$1,\
$$(patsubst $$(root)/%,-C $$(root) %,\
$$(filter $$(root)/%,$2))) \
$$(filter-out $$(addsuffix /%,$1),$2)
endef
I'll list the relevant bits of the template which lead up to this being used.
# Default if not set
$1_RESOURCES_ROOTS ?= $$($1_RESOURCES_DIRS)
# Searches for any files which aren't .java
$1_RESOURCES_SCAN := $$(if $$($1_RESOURCES_DIRS),$$(shell find $$($1_RESOURCES_DIRS) \
-type d -name CVS -o -name '.*' -prune \
-o -type f -a \! -name '*java' -a \! -name '*~' -a \! -name '.*' -print))
# Merge with any supplied explicitly
$1_RES:=$$($1_RESOURCES_SCAN) $$($1_RESOURCES)
# Build the jar
$$($1_JAR): $(stage)/$1_built
...
jar cf ... \
$(call JAR_deroot,$$($1_RESOURCES_ROOTS),$$($1_RES)) \
...
At this point I don't care about portability with the use of things like find. Perhaps I will look into guilified make in the future (i'm a firm believer in using make as the portability layer as the current maintainer is).
And some example output. The last 2 lines are the result of this JAR_deroot macro.
jar cf bin/termz-0.0.jar -C build/termz . \ -C src au/notzed/termz/fonts/misc-fixed-6x13-iso-8859-1.png \ -C src au/notzed/termz/cl/render-terminal.cl
A lot of this stuff is there to make using it easier. For example if you just want to find all the non-java files in a given directory root and that contains the package names already you can just specify name_RESOURCES_DIRS and that's it. But you could also list each file individually (there are good reasons to do this for "real" projects), put the resources in other locations or scatter them about and it all "just works".
How?
Just looking at the JAR_deroot macro ... what is it doing? It's more or less doing the following pseudo-java, but in an implicit/functional/macro sort of way and using make's functions. It's not my favourite type of programming it has to be said, so i'm sure experts may scoff.
StringBuilder sb = new StringBuilder();
// convert to relative paths
for (String root: resourcerootslist) {
for (String path: resourcelist) {
if (path.startsWith(root+"/")) {
String relative = path.replace("^" + root + "/", "");
sb.append("-C").append(root)
.append(relative);
}
}
}
// include any with no specified roots
resource:
for (String path: resourcelist) {
for (String root: resourcerootslist) {
if (path.startsWith(root+"/")) {
continue resource;
}
}
// path is already relative to pwd
sb.append(path);
}
return sb.toString();
This is obviously just a literal translation for illustrative purposes. Although one might notice the brevity of the make solution despite the apparent verbosity of each part.
Phew.
Native libraries posed a similar but slightly different problem. I still need to know the physical file location but I also need to know the architecture - so I can properly form a multi-architecture runtime tree. I ummed and aahd over where to create some basic system for storing native libraries inside jar files and resolve them at runtime but i decided that it just isn't a good idea for a lot of significant reasons so instead I will always store the libraries on disk and let loadLibrary() resolve the names via java.library.path. As it is still convenient to support multi-architecture installs or at least distribution and testing I decided on a simple naming scheme that places the architecture under lib/ and places any architecture specific files there. This then only requires a simple java.library.path setup and prevents name clashes.
Ok, so the problem is then how to define both the architecture set and the library set in a way that make can synthesise all the file-names, extensions (dll, vs so), relative paths, manifest entries in a practical yet relatively flexible manner?
Lookup tables of course ... and some really messy and hard to read macro use. Oh well can't have everything.
Libraries are specified by a pair of values, the location of the directory containing the architecture name(s), and the base name of the library - in terms of System.loadLibrary(). These are encoded in the strings by joining them with a colon so that they can be specified in a single variable. The final piece is the list of platforms supported by the build, and each library must be present for all platforms - which is probably an unnecessary and inconvenient restriction in hindsight.
This is the bit of code which converts the list of libraries + platform names into platform-specific names in the correct locations. I'm not going to bother to explain this one in detail. It's pretty simple just yuck to read.
# # lookup tables for platform native extensions # # - is remapped to _, so usage is: # # $(_$(subst -,_,$(platform))_prefix) = library prefix # $(_$(subst -,_,$(platform))_suffix) = library suffix # _gnu_amd64_prefix=lib _gnu_amd64_suffix=.so _gnu_amd32_prefix=lib _gnu_amd32_suffix=.so _mingw32_amd64_prefix= _mingw32_amd64_suffix=.dll _mingw32_amd32_prefix= _mingw32_amd32_suffix=.dll # Actual jni libraries for dependencies $1_JAR_JNI=$$(foreach p,$$($1_PLATFORMS), \ $$(foreach l,$$($1_RT_JNIADD), \ $$(firstword $$(subst :, ,$$l))/$$(p)/$$(_$$(subst -,_,$$p)_prefix)$$(lastword $$(subst :, ,$$l))$$(_$$(subst -,_,$$p)_suffix)))
Thinking about it now as i'm typing it in a simpler solution is possibly in order even if might means slightly more typing in the calling Makefile. But such is the way of the metamake neophyte and why it takes so long to get anywhere. This is already the 2nd approach I tried, you can get lost in this stuff all too easily. I was thinking I would need some of this extra information to automatically invoke the jni makefile as required but I probably don't or can synthesise it from path-names if they are restricted in a similar fashion and I can just get away with listing the physical library paths themselves.
Simple but restricted and messy to implement:
termz_PLATFORMS=gnu-amd64 termz_RT_JNIADD=../zcl/jni/bin:zcl jni/bin:termz
vs more typing, more flexibility, more consistency with other file paths, and a simpler implementation:
termz_RT_JNIADD=../zcl/jni/bin/gnu-amd64/libzcl.so \ ../zcl/jni/bin/mingw32-amd64/zcl.dll \ jni/bin/gnu-amd64/libtermz.so
Ahh, what's better? Does it matter? But nothing matters. Nothing matters.
Ok the second one is objectively better here isn't it?
After another look I came up with this to extract the platform directory name component:
$(lastword $(subst /, ,$(dir $(path))))
One other fairly large drawback of programming make this way is the abysmal error reporting. If you're lucky you get a reference to the line which expands the macro. So it's a lot of hit and miss debugging but that's something i've been doing since my commodore days as a kid and how I usually work if i can get away with it (i.e. building must be fast, i.e. why I find it so important in the first place).
And if you think all of that looks pretty shit try looking at any of the dumb tools created in the java world in the last 20 years. Jesus.
It seems i hadn't had enough of the terminal after the post last night - i was up till 3am poking at it - basically another whole full-time day. I created a custom terminfo/termcap - basically just started with xterm and deleted shit I didn't think I cared for (like anything mouse or old bandwidth savers that looked like too much effort). But I looked up each obtuse entry and did some basic testing to make sure each function I left in worked as well as I could tell. Despite the documentation again there are a lot of details missing and I had to repeatedly cross checked with xterm behaviour. Things like the way limiting the scroll region works. And I just had a lot of bugs anyway from shoddy maths I needed to fix. I then went to look at some changes and netbeans had been too fat so I went down the rabbit-hole of writing meta makfiles so I could do some small tests in emacs ... and never got that far.
I really needed a proper break from work-like-activities this weekend too, and a lot more sleep. At least I did water the garden, mow the lawn, wash my undies, and run the dishwasher.
... so after a long and full day of hacking ...
The screenshot is from an OpenCL renderer. Each work item processes one output pixel and adds any attributes on the fly, somewhat similar in effect to how hardcoded hardware might have done it. I implemented a 'fancy' underline that leaves a space around descenders. The font is a 16x16 glyph texture of iso-8859-1 characters. I haven't implemented colour but there's room for 16.
On this kaveri machine with only one DIMM (== miserable memory bandwidth) the OpenCL routine renders this buffer in about 35-40uS. This doesn't sound too bad but it takes 3uS to "upload" the cell table input, and 60uS to "download" the raster output (and this is an indexed-mode 8-bit rather than RGBA which is ~2x slower again), but somehow by the time it's all gone through OpenCL that's grown to 300-500uS from first enqueue to final dequeue. Then add on writing to JavaFX (which converts it to BGRA) and it ends up ~1200uS.
I'm using some synchronous transfers and just using buffer read/write so there could be some improvements but the vast majority of the overheads are due to the toolkit.
So I guess that's "a bit crap" but it would still be "fast enough". For comparison a basic java renderer that only implements inverse is about 1.5x slower overall.
But for whatever reason the app still uses ~8% cpu even when not doing anything; and that definitely isn't ok at all. I couldn't identify the cause. Another toolkit looks like a necessity if it ever went beyond play-thing-toy.
I got bored doing the escape codes around "^[ [ ? Ps p" so it's broken aplenty beyond the bits I simply implemented incorrectly. But it's only a couple days' poking and just 1K3LOC. While there is ample documentation on the codes some of the important detail is lacking and since i'm not looking at any other implementation (not even zvt) i have to try/test/compare bits to xterm and/or remember the fiddly bits from 15 years ago (like the way the cursor wrapping is handled). I also have most of the slave process setup sorted beyond just the pty i/o - session leaders, controlling terminals, signal masks and signal actions, the environment. It might not be correct but I think all the scaffolding is now in place (albeit only for Linux).
FWIW a test i've been using is "time find ~/src" to measure elapsed time on my system - after a couple of runs to load everything into the buffer cache this is a consistent test with a lot of spew. If I run it in an xterm of the same size this takes ~25s to execute and grinds big parts of the desktop to a near halt while it's active. It really is abysmal behaviour given the modern hardware it's on (however "underpowered" it's supposed to be; and it's considerably worse on a much much faster machine). The same test in 'termz' takes about 4.5s and you'd barely know it was running. Adding a scrollback buffer would increase this (well probably, and not by much) however this goes through a fairly complete UTF-8 code-path otherwise.
The renderer has no effect on the runtime as it is polled on another thread (in this instance via the animation pulse on javafx). I don't use locks but rely on 'eventual consistency'. Some details must be taken atomically as part of the snapshot and these are handled appropriately.
Right now I feel like i've had my fill for now with this. I'm kinda interested, but i'm not sure if i'm interested enough to finish it sufficiently to use it - like pretty much all my hacking hacked up hacks. Time will be the teller.
I stopped using gnome-terminal when i stopped working on it and went back to xterm. I never liked rxvt or their ilk and all of the 'desktop environment' terminal emulators are pretty naff for whatever reason.
xterm works and is reliable but with recent (being last 10 years) X Windows System servers the text rendering performance plummeted and even installing the only usable typefaces (misc-fixed 6x13, and 10x20, and sometimes xterm itself) became a manual job. Whilst performance isn't bad on this kaveri box I also use an uber-intel machine with a HD7970 where both emacs and xterm runs like an absolute pig whenever any GL applications are running, and it isn't even very fast otherwise (i'm talking whole desktop grinding to a halt as it redraws exposes at about 1 character column per SECOND). It's an "older" distribution so that may have something to do with it but there is no direct indication why it's so horrible (well apart from the AMD driver but i have no choice for that since it's used for OpenCL dev). I might upgrade it to slackware next year.
Ho hum.
Anyway I started poking last night at a basic xterm knockoff and got to the point of less sort of running inside it and now i'm thinking about ways I might be able to implement something a bit more complete. I'm working in Java and have a tiny bit of JNI to get the process going and handle some ioctl stuff (which seems somewhat easier now than it was in zvt, but portability is not on the agenda here).
When I wrote ZVT the primary goal was performance and to that end considerable effort was expended on making a terminal state machine which implemented zero-copy and zero-garbage algorithms. zero-copy is always a good thing but the zero-garbage was driven by the very slow malloc on Solaris at the time and my experience with Amiga memory management.
Another part of the puzzle was display and the main mechanism was inspired by some Amiga terminal emulators that used COPPER lists to re-render rows to the screen in arbitrary order without requiring them to be re-ordered in memory (memory bandwidth was a massive bottleneck when using pre 1985-spec hardware in 199x). I used a cyclic double-linked (exec) list of rows and to implement a scroll I just moved a row from the start to the end of the list which takes 8 pointer updates and a memset to clear it (and it also works for partial screen scrolls). By tracking the last row a given one was actually displayed at I could -at-any-point-later- attempt to create an optimal screen-update sequence including using blits for scrolling and minimising glyph redraws to only those that had changed. The algorithm for this was cheap and reliable if a little fiddly to get correct.
This last point is important as it allows the state machine to outpace the screen refresh rate which always becomes the largest bottleneck for terminal emulators in 'toolkit' environments. This is where it got all it's performance from.
Thinking about the problem with current hardware my initial ideas are a little bit different.
I still quite like the linked list storage for the state machine and may go back to it but my current idea is instead to store a full cell-grid for the displayable area. I can still make full-screen scrolling just as cheap using a simple cyclic row trick (infact, even cheaper) but sub-region scrolling would require memory copies - but at the resolution of 4-bytes-per-glyph these are insanely cheap nowadays.
This is the most complex part of the emulator since it needs to implement all the control codes and whatnot - but for the most part thats just a mechanical process of implementing enough of them to have something functional.
I would also approach rendering from an entirely different angle. Rather than go smart i'm going wide and brute-forcing a simpler problem. At any given time - which can be throttled based on arbitrary metrics - I can take a snapshot of the current emulator screen and then asynchronously convert that to a screen display while the emulator continues to run on it's own thread.
For a basic CPU renderer it may still require some update optimisation but given it will just be trivial cell fonts to copy it probably wont be appreciably cheaper to scroll compared to just pasting new characters every time. And obviously this is utterly trivial code to implement.
The ultimate goal (and why the fixed-array grid backing is desirable) would be to use OpenCL or OpenGL (or more likely Vulkan if it ever gets here) to implement the rendering as a single pass operation which touches each output pixel only once. This would just take the raw cell-sized rectangle of the terminal state machine as it's only variable input and produce a fully rendered and styled framebuffer as the result. Basically just render the cells as a low-res nearest-neighbour texture lookup into a texture holding the glyphs. The former is a tiny tiny texture in GPU terms and rendering a single full-screen NN textured quad is absolutely nothing for any GPU. And compared to the gunk that is required to render a full-screen of arbitrary text through any gui toolkit ever it's many orders of magnitude less effort.
Ideally this would only ever exist at full-resolution in the on-screen framebuffer memory which would also make it extremely cheap memory wise.
But at least initially I would be going through JavaFX so it will instead have to have multiple copies and so on. The reason to use JavaFX is for all the auxiliary but absolutely necessary fluff like clipboard and dnd operations. I don't really like tabbed terminals (I mean I want to use windows as windows to multitask, not as a stack to task switch) but that is one way to ameliorate the memory use multiplication this would otherwise create.
So to begin with it would be extremely fat but that's just an implementation detail and not a limitation of the design.
Still mulling that over. It's still a lot of work even if conceptually it's almost trivial.
I've added a few little things to ZCL - it's coming along quite nicely, I should probably work toward another release. I'm slowly adding the functional-like stuff into it, I decided to go with 'q.offer()' as the enqueue function, and 'ofXX()' as the factory methods. Together with the garbage collector support it does offer some interesting possibilities for code-reuse but i'm still experimenting with it in practice.
I needed to access a webcam so i added another api to jjmpeg (but i might move it) which just wraps v4l2 devices directly from the file descriptor (no library). Actually I did that a while ago but have slowly been filling it out as I needed more functionality. OTOH I started looking into the total snot-show that webcam access is on microsoft platforms and decided to give up - you can't even build the media-framework libraries with mingw-w64 as far as I can tell and you need vs of some form just to get the "system" headers. Ugh.
However I did find webcam-capture library which has already solved these problems. It's probably what i will look at as a fallback, but on linux efficiency is a bit low. The simplest webcam dump to JavaFX with my library generates almost no garbage (it provides static access directly to the driver buffers) and uses 50% less cpu time compared to using the low-level interfaces it provides despite a pretty expensive YUYV conversion step. The high level swing one is 4x the cpu overhead.
Along the way I found that was actually using videoInput "library" but I couldn't get that to link (cross compile at any rate, it should work with the ms sdk) - and in any event that just uses the directshow stuff which I had working ... i dunno, years ago. But the driver no longer works for the webcam i'm using and i'd have to buy it, ... so yeah that can wait.
And that ultimately led me to openimaj which probably would've saved me doing almost all that myself, although i would've needed to understand it anyway for OpenCL translation. And maven, ffs. But I guess I should at least have a look.
Discovery of useful software isn't that easy these days with so much noise - even if you go looking which I can't say I did ...
I've also been using a small amount of OpenGL and interoperating with OpenCL. It's become all a bit ... naff, and JOGL has been necessarily messed up to support all this nafficity. Pity I had to do this now rather than in a couple of months otherwise I would be looking at vulkan instead but with any luck that will be out soon enough to move to it before I need to get too far into GL (with the steammachines in november?). I'm sure microsoft will find some way to totally fuck-up it's cross-platform parts again though. My current thinking is that i will write a java binding for it once I have my hands on it (if only just-because) but I haven't looked into it all so far. But removing the static per-thread state should make it a lot saner.
For now I have some simple classes to do some off-screen rendering, and some OpenCL interop which is enough for what I want. Access to an output texture in JavaFX would be nice but it sounds like this is just not going to happen. Although one would expect a vulkan backend to be done at some point it will probably suffer the same hiding issues (well, with good reason I guess). If I really needed more performance i'd just use another toolkit - which is sad as that doesn't appear to be the intended vision of the javafx designers.
On that interop I had to fill out the extension mechanism in ZCL. I followed the prototype I'd created earlier. Currently each extension is provided by a different CLExtension class. It holds a pointer which in C-land is a function table resolved on the platform, and each platform object manages these. At first I was just going to use this as the mechanism for accessing extensions but it quickly becomes messy - you have to find the platform the object belongs to an in some cases this requires multiple queries (e.g. q.getDevice().getPlatform()). One approach I tried was to hide that by providing the extension methods directly on the object they extend - e.g. new CLContext or CLCommandQueue methods. These then manage looking up the extension and invoking the correct method for the given object. The details still need to be resolved but only once per object and it's all handled java-side.
There's a bit more behind the original mechanism than just code tidyiness for the extensions - they could potentially be loaded at runtime, or written separately from the core library. But on reflection how useless is that? The problem with this approach is each extension has it's own object - this is good and bad in that eventually you end up with a table required per context, queue, device, or whatever.
I think putting the extension methods on the target object is correct and after that the details don't really matter so much since it's an internal detail. But (on the fly design) I guess i should just maintain a CLPlatform reference on each object which can be extended and handle it that way. The extension objects will still be per-extension which keeps a cleaner namespace but they only need to be set per-platform which doesn't happen often. I'm pretty sure all objects have a 1:1 platform relationship, that would be the only thing to throw a spanner in the works; but the whole extension mechanism wouldn't work if that were the case.
Somewhere along this journey i came across some C++ code for something, I can't remember what it was. It was how I find most C++ code - it's been so over-engineered almost all of the actual lines of code is just boilerplate. The workings are so hidden I gave up trying to find it. It's just as bad in Java land where everyone wants to write a fucking framework before they even get started. Cut the bullshit and get to the point. C++ shows its heritage as being born from a time when "Software Engineering" was going to solve the worlds software problems by taking the programmer out of programming; using UML and CASE tools and auto-generating everything. It really shows, it is not a good language. This craziness was at it's peak just as I was going through uni and its done it's fair share of damage to the world and clearly continues to if abominations like C++ still exists.
That'll do for now.
An inconvenience in the current api is that all the enqueue functions take a lot of arguments, many of which are typically default values. This can be addressed using function overloading but this just adds additional inconvenience as there are also simply a lot of functions to overload. A related issue is things like extensions can add additional entry points which are object-orientedly resident on the queue object but placing them there doesn't necessarily fit.
And finally new compound operations need to be placed elsewhere but also fit a similar semantic model of enqueing a task to a specific queue.
So the thought is to instead to use java's lambda expressions to create queueable objects which know how to run themselves, and then at least the waiters/events parameter overload can be handled in one place.
So rather than:
// some compound task
public void runop(CLCommandQueue q, CLImage src, CLImage dst,
CLEventList waiters, CLEventList events) {
... enqueue one or more jobs ...
}
public void runop(CLCommandQueue q, CLImage src, CLImage dst) {
runop(q, src, dst, null, null);
}
public void runop(CLCommandQueue q, CLImage src, CLImage dst,
CLEventList event) {
runop(q, src, dst, null, event);
}
// usages
runop(q, src, dst, waiters, events);
runop(q, src, dst, events);
runop(q, src, dst);
I can do:
// the interface
interface CLTask {
public void enqueue(CLCommandQueue q, CLEventList w, CLEventList e);
}
// the creation (only one required)
public CLTask of(CLImage src, CLImage dst) {
return (q, w, e) -> {
... enqueue one or more jobs ...
};
}
// usages
q.run(op.of(src, dst));
q.run(op.of(src, dst), events);
q.run(op.of(src, dst), waiters, events);
This could extend throughout the rest of the api so that for example a CLBuffer would provide it's own read task factories:
public CLBuffer {
public CLTask ofRead(byte[] target) {
return (q, w, e) -> {
q.enqueueReadBuffer(this, true, 0, target.length, target, 0, w, e);
};
}
}
// usages
q.run(buffer.ofRead(target));
q.run(buffer.ofRead(target), events);
q.run(buffer.ofRead(target), waiters, events);
vs
// typical usage (without overloading) q.enqueueReadBuffer(this, true, 0, target.length, target, 0, null, null); q.enqueueReadBuffer(this, true, 0, target.length, target, 0, null, events); q.enqueueReadBuffer(this, true, 0, target.length, target, 0, waiters, events);
I think this would provide a way to add the convenience of overloading without a method count explosion. But the real question is whether it would actually improve the api in any meaningful way or merely make it different. Probably at this point it's a tentative yes on that one for many of the same reasons lambdas are convenient such as encapsulation and reuse.
There are some issues of resolving state at point-of-execution and threads but these are already an issue with OpenCL code to some extent and definitely with lambdas in general.
One could keep going:
// the interface
interface CLTask {
public void enqueue(CLCommandQueue q, CLEventList w, CLEventList e);
public default void on(CLCommandQueue q) {
enqueue(q, null, null);
}
public default void on(CLCommandQueue q, CLEventList w, CLEventList e) {
enqueue(q, w, e);
}
}
// usage
buffer.ofRead(target).on(q);
Despite this having the benefit of layering in isolation above the base api I think it starts to get a little absurd and turns into "much of a muchness" deckchair shuffling.
Although this addition is probably useful:
// the interface
interface CLTask {
public void enqueue(CLCommandQueue q, CLEventList w, CLEventList e);
public default CLTask andThen(CLTask after) {
return (q, w, e) -> {
enqueue(q, w, e);
after.enqueue(q, w, e);
};
}
}
// usage
q.run( buffer1.ofRead(target).andThen(buffer2.ofWrite(target)) );
Actually I didn't really intend it as an outcome but this also becomes a lot more usable if the resources in questions are automatically reclaimable via gc as per my last post. Whole state and work spaces can be retained and reused through nothing more than a CLTask reference.
I think i've convinced myself of the utility now but either way it takes very little code to try it.
I'd already done it in jjmpeg but i wasn't really keen on the way i implemented it there and wanted to see if i could come with a more streamlined solution. Like when I did it for jjmpeg I started with this article about JavaSE finalisation and using weak reference queues.
I think the solution I came up with will work ... and it turned out to be rather simple in the end.
Previously all CLObjects were a simple lightweight pointer handle with all the details passed to the C functions. They all have an init(pointer) constructor which was called directly from the JNI layer. Duplicate objects referencing the same resource were not an issue so I just let it happen. Well it's easy to break but if you treat objects like the C pointers they are and know that dangling references are possible then it's not unsolvable.
But for GC to work the references need to be unique. This is fairly easy to guarantee as the resources are just memory pointers - which are guaranteed to be unique and unchanging. So rather than the JNI layer invoke the constructors directly I just call a factory method with a type index which lets me move some of the code into Java - it isn't significantly simpler but it is more flexible.
For the reference queue to work properly I need to store them in a container anyway so this conveniently meshes with using a hashtable to uniqify the objects.
static CLObject toObject(int ctype, long p) {
CLObjectHandle h = referenceMap.get(p);
if (h != null)
return h.get();
return classTable[ctype].newInstance(ctype, p);
}
My first attempt passed the Class through (this is how i did it in JNI) but I changed it to an integer. It makes the JNI a bit easier and having the type as an integer simplifies the release call (OpenCL api isn't OO and has per-type release functions). Being able to identify the object fully using primitive types also lets me freely use them without polluting the reference tree; which is critically important when dealing with gc.
Now comes the bit which i fucked up in jjmpeg (well the biggest bit). Each object is represented by 4(!) classes. An autogenerated native abstract class which includes the static native method prototypes and a hand-written native concrete class which implements any type-specific dispose or construction semantics. Then there is an autogenerated abstract public class which includes all the autogenerated methods again - this time invoking all the methods on the native class after looking up the object pointer. And finally a hand-written public concrete class which includes constructors, helpers, and any other special cases where the details are better hidden.
This is just a lot of code - every public method on the "java" class ends up calling a native method on the "native" class so every method needs at least two implementations; . This was the main driver for ZCL simply using a single JNI implementation and foregoing this redundant juggling of the call stack just to insert the resource pointer into the call. In most cases in ZCL the public api is just the native method and it needs no redundant wrapper.
This time I just added a single general-purpose CLObjectHandle weak reference type which is used by all instances to track the native resource. It just holds the pointer (and the ctype) and implements the release. I just add one of these to each CLObject in one place.
public abstract class CLNative {
final long p;
protected CLNative(long p) {
this.p = p;
}
...
}
public abstract class CLObject extends CLNative {
final CLObjectHandle h;
protected CLObject(int ctype, long p) {
super(p);
h = new CLObjectHandle(this, ctype, p);
}
...
static class CLObjectHandle extends WeakReference<CLObject> {
long p;
int ctype;
CLObjectHandle(CLObject referent, int ctype, long p) {
super(referent, referenceQueue);
this.p = p;
this.ctype = ctype;
referenceMap.put(p, this);
}
void release() {
if (p != 0) {
map.remove(p);
CObject.release(ctype, p);
p = 0;
}
}
}
...
}
This and a bit of house-keeping is all that is required.
Having release be idempotent allows explicit release mechanisms to remain - for those cases where you can't afford to let the native resource management be at the whim of the garbage collector. For this reason i may also have to move the native pointer resolution in the JNI from a CLNative.p field lookup to resolving it via the handle. I need to investigate the cost of doing this first, and also whether explicit release like this will actually work in practice (e.g. if you release an object with more than one reference, does it fuck up?). Doing this would also let me use the correct integral type if I felt the need by just creating two different CLObjectHandle classes (32/64) and resolving sizes in the JNI code.
There is some potential problems where you resolve an object for the first time via a non-referencing api (for example clGetProgramInfo(CL_PROGRAM_CONTEXT) and the like) and then let the reference expire. But this shouldn't normally be a problem since you would have to get the context before creating the program and are going to be keeping it around for the lifetime of the program and thus only one xxRelease is every invoked. And this should normally hold for everything else too. If it turns out to be an issue I have mechanisms I can use to address it from adding an explicit object reference to the given objects (e.g. a CLContext to each CLProgram created), or adding phantom reference bumps on specific apis.
It's actually a devilishly difficult thing to test and verify: even once you know the exact reference counting semantics of every OpenCL api the interaction with the JVM will hide faults.
I haven't explored further but having unique objects and gc lets me freely cache local copies of resource handles for convenience or efficiency and so on. It really simplifies using the library enough as it is.
The next zcl release will include this as well as a couple of bug fixes and some other things which make it easier to use. Dunno when that might be though.
I'd had a blast of Res0gun and DRIVECLUB earlier but TV was dull so I started poking around some trivial C struct-based object system but then realised how much i'd forgotten since GObject and CamelObject. And then realised all the boilerplate that would be needed to even use such a one, so I went back to my RKRM: Libraries and looked into cloning BOOPSI instead. The only boilerplate that needs is setting a dispatch method, although the dispatch method itself ends up being fat as it fulfills the role a vtable would.
BOOPSI (basic object oriented programming system for intuition) was the AmigaOS 2 solution to general 'objects in C' which was apparently based on SmallTalk (Amiga libraries and devices are also object oriented but are not as general). Everything is implemented using a programmed dispatch call stack rather than vtables. It's not particularly fast but it is very small and flexible and it does have one rather interesting benefit not found in C or C++ - the ability to change any object in the hierarchy without a full recompile whilst still retaining single-instance memory blocks.
The short story: I got a couple of hundred lines into the code which is enough to instantiate objects and define classes together with some core support utilities.
Will I keep poking? I'm slightly curious perhaps but not quite curious enough for that as it gets involved very quickly. Maybe if I use GLX instead of the raw X I was thinking of (BOOPSOGL?). OpenVG? Text rendering is the biggest hassle either way. And layouts, although i've looked at that before.
I guess at least one observation is that back then this stuff looked so fat and cumbersome (albeit a large improvement over base intuition or gadtools), but then yeah, i've seen what else has come since and it really really wasn't.
