This will be my last post to a-hackers-craic!
I've moved all the posts and images to http://www.zedzone.space/blog, although how they look and how they work are still very much work in progress. I had already moved most of http://users.on.net/notzed to http://www.zedzone.space yesterday.
Some things might be broken for a while but everything apart from comments should have already made it across intact.
Byeee.
I haven't seen any patches for my laptop or work computers so far - one because i don't run any av software on microsoft? Ugh, what a fucking mess. I barely use that machine any more anyway, even less so while running that junk.
What's most appalling is that the problem has been sat on for 6 months under "responsible disclosure" while intel continued selling gimped hardware based on bogus benchmarks. And yet the linux and *bsd developers are either only just starting on or working on cleaning up fixes very recently - even intel is still fucking around with microcode updates.
The tech press, apart from The Register (who really broke this story) has been pretty shithouse too. "big nothingburger" for a massive massive security hole that isn't actually fixed by any of these OS kernel patches, merely mitigated. It's a big big serious problem, with big big serious costs, and the handling the disclosure has been an utter disaster - seemingly designed to spread fud about the impacts and costs. Still, if you relied on a single supplier - and particularly a nefarious piece of shit company like Intel - for any serious hardware investment, sucked in I guess? We all have to pay for your stupidity in the end regardless.
Blah.
I was insomniac[sic] late night and bored so I took the latest stack overflow survey. Apart from some pretty strange questions it's pretty clear why nobody reads my blog - no mention of almost any of the technologies I use day to day. Well at least they had the brains to identify C and C++ as two separate languages I suppose.
Some of the "order these in importance to you" questions I just skipped - I didn't care about the topic at all. A minor problem was a bit of confusion with "framework" "language", etc - where would OpenCL fit if they even included it? No CUDA either for that matter. Plenty of questions about advertising. No I don't use an ad-blocker but I turn off javascript which makes the site oh so much faster and kills most adverts as a bonus.
Sleep has been pretty miserable lately - probably a side-effect of overdoing it a bit, but I think i've gained enough weight again for the sleep apnoea to kick in. One example of overdoing it was the last weekend. Hit the beach Saturday afternoon, dropped in at a mates place - drank a carton of (bloody expensive) piss, went to a party (total strangers apart from him), was up all night whilst everyone else was falling asleep, back to his place to pick my bike up - for a couple more, another dip, and finally dropped by the pub on the way home for a fast (very slow!) one. Barmaid thought i was managing well all things considered but maybe that's because I always look exhausted and feel like shit? Nothing like a 36 hour day to get you tired; and then I was awake after only 4 hours of bloody sleep. For a couple of silly reasons I also got sunburnt to a crisp, oops.
Finally kicking into summer mode!
Well for whatever reason I got stuck into redoing jjmpeg and seem to have written most of the code (90%?) after a couple of weekends. It was mostly mandraulic and a bit tedious but somehow surprisingly relaxing and engaging; a short stint of unchallenging work can be a nice change. A couple of features are still missing but the main core is done.
Unfortunately my hope that the ffmpeg api was more bindable didn't really pan out but it isn't really any worse either. Some of the nastiest stuff doesn't really need to be dealt with fortunately.
I transformed most of the getters and setters into a small number of simple macros, and thus that part is only about as much work as the previous implementation despite not needing a separate compilation stage. I split most of the objects into separate files to make them simpler to maintain and added some table-based initialisation helpers to reduce the source lines and code footprint.
It's pretty small - counting `;' there's only 750 lines of C and 471 lines of Java sources. The 0.x version has 800 lines of C and 900 lines of Java, a big portion of which is generated from an 800 line (rather unmaintainable) Perl script. And the biggest reduction is the compiled size, the jar shrank from 274KB to 73KB, with only a modest increase from 55KB to 71KB in the (stripped) shared library size (although the latter doesn't include the dvb or utility classes).
There's still a lot of work to do though, I still need to test anything actually works and port over the i/o classes and enum tables at the least, and a few more things probably. This is the boring stuff so it'll depend on my mood.
In other news I finally killed my PC - I tried one more time to play with the BIOS and after a few updates it got so unstable it just crashed during an update and bricked the motherboard. Blah. I discovered I could order a new BIOS rom so i've done that and i'll see if i can recover it, otherwise I might get another mobo if I can still get AM2+ boards here, or just get another machine. I'll probably look into the latter anyway as it's always been a bit of a hassle (despite working flawlessly when it does and it's a very nice small machine.
Well I was up till 3am fucking around with this bloody machine.
After verifying the hardware actually works it seems that the whole problem with my RAM not being found is the damn BIOS. I downloaded a bunch of BIOSs intending to try an older one and realised I hadn't installed the latest anyway. So I dropped than in and low and behold the memory came back. Yay.
So now I had that, I thought i'd try and get OpenCL working again. Ok, installed the latest (17.40) amdgpu-pro ... and fuck. Unsupported something or other in the kernel module. Sigh. I discovered that 17.30 did apparently work ... but it took a bit of digging to find a download link for it as AMD doesn't link to older releases (at this point i also found out midori has utterly shithouse network code. sigh). I finally found the right one (that wasn't corrupt), installed it and finally ...
Oh, back to 3.5G ram again. FAAARK.
At this point the power went out for the whole street so I had a shower using my phone's torch and went to bed.
Did some more digging when I got up (I was going to give up and say fuck it, but i've come this far), tried manually adding the ram using memmap, and finally confirmed the problem was the BIOS. So i tried an older one. That worked.
But only for a while, and then that broke as well. So trying the previous one ... groan.
Mabye it's time to cut my losses, it's already 1pm, the sun is out, heading for a lovely 31 degrees.
I also got rocm installed but i don't know if it works on this hardware, although at least the kernel is running ok so far.
After posting the result I kept experimenting with the code I `live blogged' about yesterday. I did some linlining but was primarily experimenting with multi-threading. I also looked at a decimation in time algorithm (which i kept fucking up until I got it working today), and a coupe of other things too, as will become apparent.
First a picture of a thousand words.
Now the words.
This picture shows the CPU load over time (i'm sure you all know what that is) as I ran a specific set of tests on a test against 8x runs of a 2^24 complex forward transform. The lines represent each core available on this computer. I put some 2s sleep calls between steps to make them distinguishable. Additionally each horizontal pixel represents 250ms which is is about the minimum sampling time which gives usable results.
Refer to earlier posts as to what the names mean.
That's the setup out of the way, the first 3rd or so of the plot. It's important but not as important as the next bit.
Well that's it I suppose. It utilises more of the available cpu resource and executes in a shorter time. But how was that achieved?
As Deane would say, ``Well Rob, i'm glad you asked''.
It only required a couple of quite simple steps. Firstly I copied the "radix4" routine into the inner loop of "radix4_pass". The jvm compiler wont do this itself without some options and it makes quite a difference of itself. Then I copied this to another "radix6_pass" which takes additional arguments that defines a sub-set of a full transform to calculate. I then just invoke this in parts from 4x separate threads, and keep doing that sort of thing until I hit the "logSplit" point and subsequently proceed as before. It was a quick-and-dirty and could be cleaned up but probably wont add much performance.
It was a bit of mucking about with the addressing logic but once done it's actually a fairly minor change: yet it results in the best performance by far.
At this point i've explored all the isssues and am working on a complete implementation which ties it all together. I think I will write two implementations: one using fully expanded tables for "ultimate performance" and another which calculates the Wn exponents on the fly for "ultimate size". Today I got a DIT algorithm working so I will fill out the API with forward/inverse, pairs-of-real, real-input, perhaps in-order results, and a couple of other useful things to aid convolution performance. Oh and 2D of all that.
But now for a little rant.
So as part of this effort, I ended up having to write my own cpu load monitor. The only one I had available on slackware only has a tiny graph and is mostly just a GUI version of top. The dark slate blue is user time, the grey area is idle, crimson is cpu load, irq is medium sea green and the io wait is golden rod. The kernel doesn't report particularly accurate values in /proc/stat but it sufficed.
But I had intended to annotate this image with some nice 'callouts' and shadowed boxes and whatnot so i wouldn't have to write those 1000 words just to explain what it was showing. However ...
gimp has turned into a "professional photographer editing suite" - i.e. a totally useless piece of junk for most of the planet (and pro photographers wont use it anyway?). So the only other application i had handy was openoffice "dot org" (pretentious twats) draw. I even started to track down the dependencies of inkscape to build that but gtkmm? Yeah ok. But openoffce: Jesus H fucking-A-cunt-of-a-christ, what a load of shit that is. I can't imagine how may millions of dollars that piece of rubbish has cost in terms of developer hours and wasted customer time (aka luser `productivity') but i'm astounded by just how terrible it is. It runs very slow. Has some weird-arse modality/GUI update bullshit going on. Is a total pain to use (in terms of number of mouse clicks required to do the most trivial of operations). And above that it's just buggy as hell. I'd call up the voluminous settings "dialogue" to change a background colour and then it would decide to throw any changes i'd made if i didn't explicitly set it every time.
But it's in good company and about as shit as any `office' software has ever been since the inane marketroid idea to lock users into fucktastic `software ecosystems' was first conceived of. Fuck micro$oft and the fucking hor$e it fucking rode in on.
I was so pissed off I spent the next 4+ hours (till 5am) working on my own structured graphical editor. Ok maybe that's a bit manic and it'll probably go about as far as the last 4 times I did the same thing the last 4 times I also tried using a bit of similar software to accomplish a similar seemingly-simple goal ... but ``like seriously''?
To phrase it in the parlance of our time: what in the actual fuck?
Pity I can't afford to retire fully although given no male in my family has reached 60 yet that might not be true either! Probably should be though as two of those were somewhat self-inflicted and dad was cancer in the 80s.
But I can afford to quit this one if I choose to.
For a start it's been around 10 years which is much longer than i've had any other job for and whilst it's been often challenging and engaging there is always the fair share of frustration and baggage built up over such a span of time. For instance i've had years of work stuck in locked drawers unused. Stuff that should have been published, stuff that may even be published yet. It's even in an industry I don't particularly want to be a part of (not that I can think of any I do want to be part of!). On-top of all this there have been a good few years of very odd circumstances which only came to a head around xmas last year. Circumstances that while discussing with peers in hindsight appear to border on abuse. It was only though incidental rather than malicious intent but the cause doesn't affect the outcome. It's been raised and addressed but it feels too little too late since too much damage has been done that just can't be undone. Actually the initial reaction to the insight i've gained about the workplace itself in the last few weeks is that the whole joint is a complete fuckup and is a big reason i'm thinking of leaving sooner rather than later.
One other particular thing of note that crossed my mind over the last couple of days whilst reflecting on the whole period is that i've made absolutely no new friends during it - not a single one. And I don't just mean "real proper" friends you keep in touch with decades later which can be rare - I've had absolutely no social interaction with anybody outside of being at work itself (or lunch) or the annual xmas party. And the xmas party is mostly contractors i've never worked with. It's not entirely without reasons, to start with there just wasn't that many I did work with, and most people already have their lives done at this age. But it still does seem a little odd in hindsight given i've generally gotten on quite well with most of them and all the time involved.
Despite all this I've usually felt it was adequately compensated with lower and flexible work hours, an almost completely free reign in technology and implementation choices, and extended amounts of summer leave (even if it was usually due to funding shortages). But even these - as absolutely awesomely awesome as they are to me - have their limit of worth.
So I haven't decided yet.
One reason is of course what happens next. I don't really feel like i'm capable of much at all right now (look at the dearth of hacking on this blog for evidence) but that might be more perception than reality and it could change in the blink of an eye. I can however afford to take plenty of time to get my shit together if I need to (decades actually). Another is that I've never had to actually look for any job i've ever had and i'm generally pretty hung up about doing such things for the first time, yeah still. Due to working for Ximian/Novell and now these guys I don't have much of a local 'network', although it's not empty either. And my skills and interests are (probably?) not in much demand around this sleepy city outside of where I am now. Well obviously with my skills and experience I can code almost anything for almost anything (or quickly learn how to) but there's a lot of rather unappealing anythings out there that get coded and there's often some really poor technology taking up the flavour-of-the-month pedestal at any given time as icing on the cake.
If I decide in the positive will I manage to last out the remaining money for this year? Professionally one would think there is no question but personally once a decision like this has been made it will be hard to continue. I'm just a casual employee (no notice needed) and this is not an entirely professional matter I think. Another 3 months? Hmm.
I'll think about it some more I guess.
update: Told 'em. Well, we'll see. They don't want to let me go. I mean really not. I guess i'm worth something.
I did find some (pretty crappy) sort-of-actual-shorts (as opposed to no-they're-actually-longs, you know, they actually end above the fucking knee) because the typical length shorts of the day are too hot, too uncomfortable, and rub the top of my legs in ways that cause soreness. Now I don't have such a fat-arse at least it's possible to have them fit the leg when they fit the waist. One of the pair I got is basically white - they look like 60s sports day shorts, I just need the white singlet and dunlops and the no 1 haircut to go with an old family portrait, but they're just for for around the house. Either the sizes are a little "generous" or i'm down to a small now and that size definitely wasn't tight.
Actually I haven't really stopped losing weight despite eating pretty normally and doing enough basic exercises to gain muscle. This week I measured a minimum "under 69.0" for the first time in decades (but that was just the extreme of the trough). I somewhat expected it to level off by now.
I went looking for a battery for my bike headlight - the charger for the chinese junk I bought broke but the light is fine and i've been charging the mystery battery with a PSU I made in uni - but i got really pissed off at the dickhead at jaycar who couldn't do more than point at the shelves I'd already looked through so i gave up and walked out. If i just want to get their cheap junk i can get it myself, i only went to the shop to see if they had knew anything I didn't and to support a local business. dx.com it is I guess. Last time I tried the battery bar in the city but the shop assistant was utterly clueless and the 'tech guy' was a condescending fuckwit and it seems more like a "junk bar" than a battery bar anyway.
There's two local shops I will NEVER return to ever again. I made the effort to go to a local shop physically and with a literal pocket full of cash, and they weren't interested in even attempting to close any sort of sale. Fortunately at the expensive wool-clothes shop I went to after the battery bar I had a long chat to the bored shop assistant and she tried to upsell everything 'male' they sold (which isn't much but i hadn't intended to get a jumper when i went in).
I bought some dried fruit from The Market and tried to find a silicone gasket for a (stove-top) coffee maker for one of my brothers - but nobody sells them. At least one guy said it was because they never wear out - which is the whole bloody point! I got one for my machine at Gaganis Bros but they're still in the bad books for their bicycle parking arrangements they didn't have many back then anyway.
At this point i was ready to fall apart or have an accident (i've just been in a foul mood all week and this made it worse) so I said fuck it and had a beer at the Exeter. I quite like the pub itself and the staff are decent but this isn't the first time i've looked around and realised I was surrounded by other middle-aged men so I decided if I have any more I should try somewhere else. I decided at this point to go buy some bike shit after seeing it last time - I grabbed some new shy shorts (30 was the right fit, so yeah I guess I am small now) - the ones i've been using lately are 36 and they were too tight last year but now they feel like nappies with their built-in chamois and the draw-string. Also too long and too hot. Ugh I spent like 170 on another pair of cargoish things at the same time back then which I had been using but they feel like clown pants now, they were even bigger (and both have faded poorly, why are they all black anyway??). And a couple of jersey's which are possibly a bit on the tight side but we'll see - most they had were black and that's too hot so I got a white one and a red one to match my roadie (hey why not). One specific jersey I used most recently still has serviceable cloth but got stretched too much to be passable so at some point I will see if i can take it in somehow. Those expensive bike faded shorts are too big for that but I might also try to take in some other stuff I've never even worn if i can get my sewing machine to cooperate (stuff i was too fat to fit, and now falls off).
So that bike shit (incl, shoes I bought last week, new pedals and cleats), the wool tshirts and jumper, the "POW" PE shorts, jerseys and shorts, a new tyre, well over a grand. It is what it is.
I ended the afternoon with a few beers at the Austral - fuckit, if i get gout so be it (but so far so good?). I had 4 total for the day but I think 2 was probably what I should've stopped as after those two I just wasn't' really feeling anything good and all it left me with was a headache and a churned guts which still persists a couple of hours later.
I contemplated life and it came up very very short. As ever.
I even thought about tattoos. Why? To belong to "something" I guess? I didn't think about them too much.
First issue I looked at was that I broke OpenGL when i upgraded to the latest catalyst driver. I tried various kernels, even a long-overdue update of slackware (which lead to a another few hour's diversion earlier in the month trying to un-fuckup firefox as much as i could), using the slackware kernel, etc. But finally I used strace and slackpkg to determine it was using a stale .so left over from AMD's previous abomination of an install script so once that was deleted all was well. SVM still wont work in any of my code - which is still baffling as it works from the AMD SDK samples. I should really post something on the AMD forums but i haven't used it for a while.
The other problem has been only half my RAM started showing up a few months ago; the bios shows 8G but linux only sees 4G. This is ok for what i normally use the machine for but netbeans brings it to it's knees if you have a couple of projects open. I thought it was down to a faulty DIMM and If i hadn't forgotten and needing to come home earlier yesterday I might have bought some. But I tried each individually and they seemed to work - passing the linux make test at least. I tried lower speeds, voltages, all sorts of things. It appears a fairly common problem but nobody had much of a solution.
I even upgraded the bios - which was another couple of hour diversion as that cleared the EFI boot record and I had to re-figure out how to re-install it. I used a USB-bootable tool-linux distribution called grml - very impressed - I just dd'd it onto the usb stick, and it even provides an EFI boot record so i could just use efibootmgr to re-add it. I know i've tried puppylinux multiple times before but it never worked so I might get a usb stick just to keep this handy.
The last thing I tried was dusting the sockets and then trying to jiggle the cpu in it's socket - I don't have any thermal paste so I couldn't get at it properly but i wiggled a bit without freeing the heatsink.
Now ... that appeared to work. I did some tests and so on and it seemed to be ok, ... I even wrote a post about it.
But then I had to tinker, so i was playing with the memory speeds - to see if "faster" ram really made any real difference - but I broke the whole machine so i had to power cycle it a few times till the BIOS reset everything, ... and yeah back to 4G of ram. Blast. I swore at it, shut it off, and went to bed early.
On a cold boot I did get 8G out of it but then the system crashed while I was using it, so something is amiss. It could still be the RAM I suppose but at this point i'm more inclined to believe it is the motherboard or PSU. I don't feel like investigating further for the moment and will just see how it goes (i just cold booted and it's 8G again *shrug*).
However, some good did come of all this.
When I was playing with kernels I went to the trouble to actually go through all the config options and fuck off everything I didn't need. It's down to 3.2M packed including my system filesystem with no module so i don't need an initrd, and it boots a bit faster (not that it was any slouch). I'm also trying the fully-preemptive kernel and i'm liking it so far, even under very heavy load the system remains interactive such that you barely notice at all. Hmm, I just noticed the sound mixer is missing, NM.
But the really good bit is now the CPU runs almost cold - i don't know if it's the BIOS upgrade, the kernel customisation, or some BIOS setting that changed (i already had it on 45W TDP because it got too hot), but the difference is marked. Previously any high-load task such as a 'make -j4' would cause the fan to kick in almost immediately and if i didn't also up the case fan speed (manually) fairly promptly the whole desk would take off. I've currently got a kernel build running for a few minutes triggering a load-average of over 8, with the case fan on it's slowest setting; and the CPU fan hasn't throttled up at all.
Maybe the machine is just running a lot slower now - but I can't really tell so what does it matter if it is.
Update: it got unstable so i took one dimm out. Seems ok so far. Maybe it's just the PSU? It seems like the most fragile component and it's almost certainly underpowered anyway (sigh, a poor decision that one). I dunno. I started looking at new cases & psus but if it remains stable albeit in a reduced capacity I'll be in no rush ...
So I worked out what the mis-feature was actually called - it's called "show method parameters".
Then I downloaded the netbeans source code (all of it, i used a zip - netbeans-src, not sure if i could've just got the ide module, mercurial looked like it was going to take forever so i didn't bother).
I worked out where the offending bit of snot lived: "editor.completion"
I fixed it:
--- editor.completion/src/org/netbeans/modules/editor/completion/CompletionImpl.java~ 2014-11-18 19:07:58.000000000 +1030
+++ editor.completion/src/org/netbeans/modules/editor/completion/CompletionImpl.java 2015-05-02 22:39:20.935817693 +0930
@@ -1271,16 +1271,6 @@
* May be called from any thread but it will be rescheduled into AWT.
*/
public void showToolTip() {
- if (!SwingUtilities.isEventDispatchThread()) {
- // Re-call this method in AWT if necessary
- SwingUtilities.invokeLater(new ParamRunnable(ParamRunnable.SHOW_TOOL_TIP));
- return;
- }
-
- if (ensureActiveProviders()) {
- toolTipCancel();
- toolTipQuery();
- }
}
/**
And then after a few false starts trying to work out how to compile the module, I got it compiled. I put "cluser.config=ide" in nbbuild/user.build.properties and then ran 'ant jar' in the editor.completion directory. I'm not exactly sure what made it work in the end because things sort of failed and then they didn't. It refuses to build on jdk8 (as documented) but i had a jdk7 lying about already. It didn't take very long - i was genuinely impressed.
I found where the generated module went (nbbuild/netbeans/ide/modules/org-netbeans-modules-editor-completion.jar) and then copied that into both the system (/usr/local/netbeans-8.0/ide/modules/) and local (~/.netbeans/8.0/modules) module dir.
And then I started netbeans and checked to see if it took. So far looks ok.
It's a pretty strange feature to start with given it's functional overlap with code completion but it's a mystery why it is also turned on automatically every time you do any code completion or parameter lookup.
For starters the code completion window already shows this information - why pop up another less readable tooltip to duplicate it?
The other problem is that it shows the parameter lists for ALL functions of the same name not just the one specified by the current arguments (or even the type of the argument under the cursor). This just makes it a big mess with insufficient local context to make it readable at 'thinking speed'.
And the icing on the cake is that once you've 'done' your completion or parameter lookup task and moved the cursor - it decides it still wants to hang around a bit longer and jumps to the parameters of the function call you're inside of - i.e. if you're in an in-line lambda or a anonymous class definition it will move up to the function call that the lambda or new Thing is a parameter of. i.e. that code you already finished, perhaps months ago.
So ... out comes ESC. And again, and again. Almost EVERY TIME I use code completion. When i'm cutting lots of code this can be up to dozens of times in a given minute. Of course I type ctrl-space to initiate it that often too, but that's something I asked for and not some undecipherable clutter to piss off.
I already have all the popups and tooltips turned off and only call them up explicitly using ctrl-space; it got to the point there was so many little boxes flashing across the screen as i typed it was like standing around while flies keep landing on your face - once or twice is no big deal but constantly it becomes more than just a bit annoying. Swatting flies is a very apt description of using some ide's in their default configs.
But given it was so easy to build I may look at another annoying behaviour. The 'show matching parenthesis' function shows the matching parenthesis of any bracket the cursor is merely touching and gives no indication of which side of the bracket the cursor is on. This just makes it harder to work out where the cursor insertion point is when you're otherwise not interested in the matching bracket.
It could just be familiarity (although i only recently turned it on in emacs so it really isn't) or it's lisp implementation but the way emacs works makes it more useful. It will only highlight the matching bracket if the cursor is to the right of a close bracket or sitting on an open bracket (insertion point is to the left). If it's both, it will highlight the opening bracket of a close bracket since that's more useful. emacs uses a block cursor rather than a line which I also find easier to use for fixed-width character editors for a couple of reasons including that it goes hollow instead of vanishing when the window loses focus - which makes it easier to relocate if you're switching between windows on the same screen(s).
I wont link to the article but i came across one that included some code that used streams everywhere for some examples. Apparently it's "obvious" how using streams simplifies the code and so on.
For a start the code isn't really any simpler; it avoids needing to calculate the output size but that isn't a very complicated calculation. Unless all you knew was streams it certainly isn't any more expressive or concise either.
I converted one function directly to arrays and it was shorter and runs at twice the speed for a given test case. It also uses about 1/6th of the memory and roughly 30 000 fewer memory allocations (oh my poor breaking gc, still, java alloc is rather fast isn't it?).
Apart from that ... oh boy it does some bad bad things to a poor innocent atomic counter.
My first thought was "very poor and unscalable use of atomic counter". Then I looked closer.
// names changed to protect the innocent
AtomicInteger count=new AtomicInteger();
int bob[] = foo.stream()
.map(f->{
Thing t=list.get(count.getAndIncrement());
int p0=f.a; int p1=f.b; int p2=f.c;
int t0=t.a; int t1=t.b; int t2=t.c;
return IntStream.of(p0, t0, p1, t1, p2, t2);
}).flatMapToInt(i->i).toArray();
So what this is attempting to do is iterate through two and merge pairs at matching indices into a flattened array of integers.
Except one of those arrays has been forced into a stream (for no reason other than it can be done it seems) - thus losing the position information. So to "recover" this information so that it can be correctly indexed into the other array an atomic counter has been added. But this solution is both dangerous and confusing. It's confusing because it looks like it should be concurrent code - that's exactly what atomic counters are for and also a common desired side-effect of using streams, but this loop is not being invoked in a concurrent context. It's probably there because it was left-over from such an attempt. It's dangerous because it is vanishingly unlikely to actually work if it actually was invoked on a parallel stream because atomic counters by definition just don't provide the required constraints and thus there is no guarantee of obtaining matching pairs. At best it looks like a thoroughly cheeky and worst-practice approach to work around the final rules for lambdas.
Whatever it is, it's sick. Don't ever do this (in any language).
In the source example this is part of a larger function where the previous two loops generate one each of these lists independently and in-order and then after this merge they are simply discarded. i.e. there is not any practical reason for any of this intermediate garbage creation apart from saving a little arithmetic in pre-calculating the required size of the array required. The two loops could be retained and just write interleaved results and without losing the expressiveness of the implementation.
But for arguments sake if you really did want to merge two array lists of a known length to an interleaved integer array, here's one approach that has worked for a few decades and is still about as good as it's going to get:
int[] bob = new int[foo.size() * 6];
for (int i=0, j=0; i < foo.size(); i++) {
Point2D f = foo.get(i), t=list.get(i);
bob[j++] = f.a; bob[j++] = t.a;
bob[j++] = f.b; bob[j++] = t.b;
bob[j++] = f.c; bob[j++] = t.c;
}
Lets also say for arguments sake that the stream example did actually work in parallel, and you would gain anything from it (hint: you wont, see the end), so you got that for free right? What about that ancient and daggy old for loop?
int[] bob = new int[foosize()*6];
IntStream.range(0, foo.size()).parallel().forEach((i)-> {
int j = i*6;
Point2D f = foo.get(i), t=list.get(i);
bob[j++] = f.a; bob[j++] = t.a;
bob[j++] = f.b; bob[j++] = t.b;
bob[j++] = f.c; bob[j++] = t.c;
}
One argument for using streams is code-reuse - which falls over once you have side-effects and so the initial example is no better than the last in that respect.
If you really absolutely positively must use streams for this because of whatever reason (there are some legitimate ones): write a proper spliterator and use StreamSupport.stream() to turn it into a stream. It will have to take the two lists in it's constructor and iterate over a container object which holds the matching pair (like Entry<,>).
One will note however that for example the linked list spliterator just breaks the iterable into batches of arrays which are then spliterated over. Whilst this conceptually may seem that it could be a win due to locality of reference and doing the work piece-meal: in reality the spliterators are run to their limit before anything starts for scheduling purposes. So all it's really doing is breaking a single allocation and copy loop into many (sqrt(n)) smaller ones; which is always guaranteed to run slower and use less memory. i.e. you could just flatten both lists to arrays and then use the per-item or array methods above and get the same result - more efficiently - and with less programmer effort.
So whilst streams can save a lot of effort in writing concurrent code; it still many of the same gotchas that can introduce performance side-effects for the ignorant. For example if you're using thread synchronisation primitives at all in any stream processing chain including custom spliterators or collectors then you're literally "doing it wrong" as:
the entire point of the parallel part of the stream framework is to exterminate this unscalable approach
(I thought it needed a bit more emphasis than em/strong/underline could provide :)
I couldn't leave it there so I tried implementing an "Entry" spliterator: one that takes two streams and passes each one into the stream as an Entry.
Because you really don't want to run this on a linked list i forced it to take arraylists. But there are few cases (random deletes or deletes from the head) where you would want to use linked lists and the container approach to linked lists creates pretty shit lists since you lose the ability to delete randomly in O(1). So even when they might be a win for the name of the data structure they often aren't due to the implementation details. But I digress.
So using a spliterator which is under 10 lines of significant code:
bob = StreamSupport.stream(new SpliteratorEntry<>(
(ArrayList<Thing>) foo,
(ArrayList<Thing>) list), false)
.map(e -> {
Thing f = e.getKey();
Thing t = e.getValue();
int p0 = f.a; int p1 = f.b; int p2 = f.c;
int t0 = t.a; int t1 = t.b; int t2 = t.c;
return IntStream.of(p0, t0, p1, t1, p2, t2);
}).flatMapToInt(i -> i).toArray();
It doesn't make any appreciable difference to the running time or to those underwhelming allocation overheads; but at least it now allows for a reusable function and it isn't just plain "wrong".
FWIW making this concurrent just makes it a bit slower while using more cpu cycles and energy. But I could have guessed "not worth it" beforehand since it just isn't doing enough work to compensate for all the overheads on such a small problem (a few thousand items).
Going back to the ADSL did however let me confirm that there's something not quite right with it, as I managed to get some multiplayer DRIVECLUB games in - whilst it wasn't perfect as their servers are still pretty shitty at least it worked a few times - and it's never worked even once using the NBN. To confirm I've flipped back and forth a few times and it's still the same: ADSL works sometimes, NBN never works (it might let me start but always kicks me out). Even using the same shitty fritz-box; so maybe it's not the modem but reports elsewhere suggest it's a piece of junk. Why internode recommend them is beyond me, although i'm sure a firmware update could fix it if it's just that.
I'm not particularly annoyed that DRIVECLUB doesn't work itself because i'm just not into that kind of competition (albeit there is a small spark when it works, but wears thin quickly against deaf and dumb strangers) but it shits me that i've got something that doesn't work properly. I reached level 50 a few days ago - I don't understand when people call the game unapproachable, as it's easy to just work at your own pace and you can still eventually complete the game. I'm very much middle-of-the-road on all the leaderboards and "challenges" and aren't even playing that much each week.
When I feel like blowing an afternoon I'll try setting up a router+dhcp+etc on a GNU box and see if that will work properly just to confirm it's the router. From what I can tell my old billion VG wont support PPPoE without going over the ADSL link - I tried but failed to get anywhere.
Gout kicked up again a few days ago but i'm not really sure why this time. I caught up with some people over a few beers a couple of times and so on with no ill effects but then a week later it just started again. All I can think of is not enough water one day or climbing up a tree barefoot to do some pruning (clumsily). Or the attack is just more delayed than I thought and it was due to something I ate/did a few days earlier. It's mostly ok unless I try to walk but I had to go to the shop today because I ran out of drugs and that was pretty limpy - and to my shock the supermarket was out of ibuprofen, although fortunately the chemist is next to it. Going to be fun getting my bike shoe off. The rate slowed but i'm still losing weight - I had my appetite back there for a bit but it's deserted me again although I'm also putting in a conscious effort too. Come on 80kg, just 2 more to go.
I'm avoiding the news and barely watching tv (working, reading, or playing games instead) but what little I pick up is probably more depressing than these localised issues. But if someone like tony-"fucknut"-abbott can be voted in by the will of the people then what are you gonna do? I just got a mail from the FSF about the TPP ... i just can't believe these slimy cunts who would sell their own nations souvereignty out for the benefit of those who least need it. Why aren't they in a gaol cell or hanging from a tree for such obvious treason?
Well writing this blew a couple of hours in-between doing the washing and some vacuuming; although it's raining now so I gotta find somewhere to hang it. But what to do with the rest of the weekend ...
Update: Internode ended up calling me one night, sort of to appologise for the delay and to follow it up. Guy was a bit annoying, you know the type trying to be `helpful' whilst constantly pushing the conversation forward and not admitting to anything. When I mentioned DRIVECLUB he just did a web search, yeah like i didn't already try that. I never wanted to have to contact support in the first place and hopefully I don't have to again.
Foot's been worse. I have no idea why as right now i'm not really eating enough for it to be diet related and i'm drinking a normal amount of water. Unless it's the weight loss itself, which I guess is just a necessary cost since I need to lose it so i'm taking advantage of the fact I don't feel hungry most of the time right now.
So i'm using my old ADSL - the one I thought i "switched" from; but alas they just kept both running and billed. I think it's because i used their crappy on-line form to request it and it's been a drawn out collection of misunderstanding and mistakes since. For an internet company they seem pretty poor at doing everything over the internet.
So I got hit with gout a couple of months ago as a nice birthday present which pretty much screwed up any tentative plans I had for that let alone the future. After the first couple of days of pain it just turned into another sore foot; I can hobble a bit by putting weight on the outside edge but even walking around the garden too much has consequences. Surprisingly, cycling is worse than walking and may hint at the trigger. It sometimes swells a bit but without shoes that isn't very noticable. A couple of weeks after an attack it starts to get to the point I can ride a few km or go to the supermarket but it still gets sore afterwards. For the most part even at it's worst the pain was mostly annoying - certainly painful but still able to sleep with drugs - although after the second attack I had to go to the shop to get groceries and walking in shoes was one of the more painful things i've ever done to myself; I should've waited for the ibuprofen to kick in properly and probably just gone bare foot. As I live alone I would have had to go eventually though.
What is most frustrating about it is that after a couple of months of summer doing pretty much nothing i was ready to get a bit of exercise and mix with humanity again; so i went for a 2 hour ride (took it mostly easy but pushed a little for the exercise), got drunk at the pub, then came home and had some biscuits and cheese for "dinner". Then suddenly it felt like i'd broken my big toe and i wasn't really sure I hadn't ... But I knew instantly it was gout (i think i'd looked it up before due to my other foot trouble). And that was pretty much summer and exercise (and life?) all over. Getting so bored sitting around at home is a big reason I started hacking code again. We also had a notably mild - and completely dry - summer this year so it was doubly frustrating I couldn't really enjoy it. As an aside some of my initial searching turned up a similar story of a guy the same age as me going for a "moderate" bike ride before his first attack.
It was a pretty depressing read looking up information about it on the internet: pretty much everything i eat was "off the list", pretty much everything i do for fun was "off the list". Then I started to notice it was almost all exclusively old wives tales full of completely contradictory information. Even the limited medical information is contradictory: e.g. it's all about the purines, well, except when they're in vegetables when they apparently don't count. Even beer which is the worst thing for gout doesn't have much in the way of purines. Also surprising that there seems to have been so little in the way of hard studies done for such a common ailment. What few medical suggestions are available seems to be targeted at the obese: it is unclear for example whether sugars, fats, and oils are bad for gout or just bad for fat people and being fat is bad for everything. I would consider myself overweight but plenty wouldn't these days due to their distorted sense of "normal" and efforts not to "offend" (typical family greeting however: when did you turn into such a fat cunt? - as it should be). The recognition in the irony of suggesting lots of exercise when you can't even walk or cycle is not terribly comforting.
So my initial readings pretty much put me off eating almost everything at the start; and now i've lost most interest in food. When I went to a doctor a couple of weeks afterwards he said meat should be ok but I've barely had any since; doing a roast just seems like asking for trouble. Pangs of hunger come and go but are easier to ignore, and small meals and snacks seem to fill me up much faster and for longer. I've lost nearly 1Kg/week so far so now i'm interested in seeing how far I can go. At this rate i'll break through 80 in another month and if i can do that at least it will be something gained from this sorry episode.
So now after a bit of time and experiments i've kinda worked out that tons of water is about all I can do to help. Together with ibuprofen if it's too sore or swollen. Conversely the main problem seems to be dehydration or more particularly a lack of urination. I usually drink plenty of water unless i'm too busy to notice, especially when cycling, but I also sweat profusely when doing anything physical so maybe it just isn't ever enough. I can have some wine and beer if i drink a lot of water - but it often gets to the point of being only slightly intoxicated while being waterlogged so it isn't remotely fun. If i feel it coming on (a pricking on the toe) absolutely drowning in water seems to prevent it going further but doing that together with visiting the dunny all night is starting to get pretty obnoxious. I'm having tons of coffee - some evidence suggests it reduces the chance of gout but again no study to say if it helps it - but at least it's something i can safely imbibe (i don't really get any buzz from it; it stops me sleeping properly if i have it too late is all). According to the internets (green) tea is supposed to do wonders but the medical evidence says otherwise: i concur with the latter. It does however have one important potential use: it lets you drink much more water without exploding. I prefer green to black without milk anyway, and black with milk doesn't seem to agree with me anymore. The doctor said no red wine but red seems to be better than white and wine better than beer. I really miss being properly beerly-drunk.
No doubt the beer and food over a lifetime helped it on it's way but I think the trigger was probably dehydration. In that 4 hour period of the bike ride+pub I had ~2L of beer and ~2L of water but only urinated once I got home. Maybe i've also just got a naturally high level of uric acid. It's unlikely to be family related; they all drink like fish - more often and intensely than I do.
I had another attack Thursday night and didn't feel like fighting it with water. It ended up fairly mild but my foot's still swollen and it feels like it's reset it's 'state' to two weeks ago. It's not like I was walking around properly anyway so hobbling a bit more is quickly becoming a matter of "so fucking what".
Since i'm pissed off with my internet too, despite having access to it, it's probably time to sit in the garden reading as i've done the last 2 days. With a bottle of water. And maybe something a little stronger - in small amounts :( It also needs a mow and other maintenance but my foot needs more rest.
The original:
Standard histogram equalise (its slightly different than the gimp due to some range handling but it's equally just as shithouse):
My other really cheap tunable histogram equalise based on truncation on the default setting. Maybe a bit dark in the dark but it is tunable further.
The new one tries to solve the same problem but in a completely different way:
And another test image which probably shows it even better:
Here the standard operator just makes a pigs breakfast of pretty much everything:
The simple limit works pretty well here:
And here I think the new one pips it. It retains about the same amount of detail with in the coins but adds more contrast to the surface they're resting on without burning it out.
The previous algorithm applies a hard ceiling to the histogram bins after they are calculated based on the mean.
This algorithm instead applies a localised sameness test before they are added to the histogram and they are simply not counted if it fails the test. This means it simply drops any areas of similar or slow-changing intensity out of the histogram calculation. There are some tweaks needed to handle extreme cases but as can be seen it works pretty well as is on challenging input.
But its somewhat more expensive to calculate so the previous algorithm is probably more desirable given the similarity of the results.
I just snarfed the base pics from over here.
Hmm, 2am again. My fibre NBN was down again tonight and I discovered my old ADSL is still connected and still being billed. Yay. What the hell internode, make it bloody clear that "switch plans" isn't a switch, if it isn't a switch. This whole thing really hasn't been anywhere near as smooth as it should've been; then again they charged me for permanent dial-up for about 6 years after i got ADSL so they certainly have form.
Probably should've gone to bed after that and a bit of TV but it's been a long week of trying to decipher some odd(?) C++ which wasn't very pleasant and I needed to 'relax'. Although the C++ thing end with some success.
I wrote a fairly long rant against it yesterday but decided not to post it. But in brief it started with the pleasant surprise of the performance of jaxb on a 200MB xml file (hah, not memory performance, but running-time), another in discovering that you can put a whole array of primitives into an attribute which drops the verbosity considerably - 200MB was after finding this out so it could've been much worse. A realisation of self-stupidity after spending an hour or so writing a s-expression loader from scratch for a completely one-off, throw-away data-transfer job (wasn't thinking straight there). Two lines of calling jaxb and a few annotated holding classes was only about 3x slower at reading the xml than the C++ all-hand-written all-in-header-files macro-template-stuff was at reading it's own proprietary binary format. For the task that would probably do but I want to run this a few times and 5 seconds is too long, so that lead to an ensuing of mirth once I tried using the java default all-i-did-was-add-an-interface serialisation via Object*Stream: about 5x faster and 30% smaller than the C++. And finally some satisfaction in porting some number crunching algorithm across and having it run 50% faster, with simple plain code using consistent syntax that compiles in a 10th of a second rather than 30, and no multiple-screen error messages for using the wrong type of bracket in one place.
Not that C++ programmers have writing needlessly complex code all to themselves, it seems to be a universal skill and one widely practiced. C++ does seem to attract adepts of the highest skill in the art though and fortunately for these code-fiends it has the most facilities to aid them in their vitally important work. They certainly seem to take pride in it. Java is probably a close second truth be told, but probably for very different reasons. The whole weird self-help like book/training scene and probably just a lot of newbies who don't know better, at least at one time (any time i look back ~5 years i always see things I wouldn't do that way now, or worse).
Now I remember one thing I thought of whilst writing the post yesterday based on my recent C++ exposure.
The C++ language just doesn't seem to understand what abstraction is for. It seems so busy hiding as many actually important machine-level details as it can whilst exposing in intricate detail as much fluffy non-essential information about it's own type system so the compiler has any hope of understanding what you intended together with some vague (and basically worthless) guarantee that all this noise will produce somehow "optimal" code - for some interpretation of "optimal" which doesn't seem to take into account physical hardware `as such' (not surprisingly given the preceding paragraph).
When these features are used in libraries (or perhaps conglomerations of syntax more succinctly described as "the headers are a platform") the information hiding can be taken to an absurd level: they've actually managed to hide the ability to understand even the syntax of a single line of program code in a widely used language by just looking at it. Now not only do you need to know the basic syntax and semantics of the language itself, you also need to have an intimate knowledge of the entire platform included in the header files before you know the meaning of basic expressions. Even in the one place where semantics cannot be deduced without extraneous knowledge - i.e. function invocation - C++ allows the removal of the one small mnemonic you had as a guide. Just think of all the man-years spent getting this snot to compile.
But back to the point - by doing this you haven't hidden anything - you've just exposed everything. Because without knowing everything you can't understand any of it.
Maybe it was a specifically weird of code but i did some searches and it seems the things it does is common and even seen as desirable. But people also believe the strangest things and the internet is on average simply full of shit.
So i checked measuring spoons ... and NONE i found in the department stores or kitchen-shit shops were to Australian standard; they only had euro/uk 15ml tablespoons.
Whilst the teaspoon is the same size at 5ml, Australian tablespoons are 20ml.
It's just baffling. I didn't raise it with the staff because I wanted to check my memory was correct first (although i was quite certain as i'd had a somewhat robust discussion with my sister over the fact about a year ago!).
Obviously I could just use 4 teaspoons or some other mix but obviously I can also choose not to buy incorrect measures from a local retailer. Maybe the supermarket will have something correct; they still SOME Australian stuff and not just (dumped) EU brands.
Update: The only ones at the local foodland were actually correct. Although "curiously" the half tablespoon is 7.5ml rather than 10ml; but nobody seems to use dessertspoons anyway so i'm not sure what it's supposed to be.
Patent trolling is usually the last gasp of a failing business. Which implies that despite (or because of) their overpriced hardware they are failing as a manufacturer; GPus are now commodity items and the margins no longer exist to run their type of high-margin business.
Patents are a cruel abomination which distort the workings of a "free market"; they directly codify rentier behaviour which costs society both economical and technological progress. The only beneficiaries are the unproductive leeches of society at the cost of everyone else.
If you're an engineer or scientist who is currently using or considering cuda for your work I suggest you reconsider both to protest this failure of a strategy and to protect the future value of your work.
Just for nvidia to consider this strategy shows they are not long for this world and choosing to use such a single-supplier would be foolhardy.
typedef float float4 __attribute__((vector_size(16))) __attribute__((aligned(16)));
void mult4(float *mat, float4 * src, float4 * dst) {
dst[0] = src[0] + mat[0];
}
notzed@minized:src$ make simd.o arm-linux-gnueabihf-gcc -c -o simd.o simd.c -O3 -mcpu=cortex-a9 -marm -mfpu=neon notzed@minized:$ arm-linux-gnueabihf-objdump -dr simd.o simd.o: file format elf32-littlearm Disassembly of section .text: 00000000I thought that the store/load/store via the stack was a particularly cute bit of work, especially given the results were already in the right order and in adequately aligned registers. r3 also seems a little too popular.: 0: f4610aef vld1.64 {d16-d17}, [r1 :128] 4: ee103b90 vmov.32 r3, d16[0] 8: edd07a00 vldr s15, [r0] c: e24dd010 sub sp, sp, #16 10: ee063a10 vmov s12, r3 14: ee303b90 vmov.32 r3, d16[1] 18: ee063a90 vmov s13, r3 1c: ee113b90 vmov.32 r3, d17[0] 20: ee366a27 vadd.f32 s12, s12, s15 24: ee073a10 vmov s14, r3 28: ee313b90 vmov.32 r3, d17[1] 2c: ee766aa7 vadd.f32 s13, s13, s15 30: ee053a90 vmov s11, r3 34: ee377a27 vadd.f32 s14, s14, s15 38: ee757aa7 vadd.f32 s15, s11, s15 3c: ed8d6a00 vstr s12, [sp] 40: edcd6a01 vstr s13, [sp, #4] 44: ed8d7a02 vstr s14, [sp, #8] 48: edcd7a03 vstr s15, [sp, #12] 4c: f46d0adf vld1.64 {d16-d17}, [sp :64] 50: f4420aef vst1.64 {d16-d17}, [r2 :128] 54: e28dd010 add sp, sp, #16 58: e12fff1e bx lr notzed@minized:/export/notzed/src/raster/gl/src$
I guess the vector extensions to gcc just aren't finished - or just don't work. Maybe I used the wrong flags or my build is broken. It produces similar junk code for the epiphany mind you. I've never really tried using them but after a bunch of OpenCL in the past I thought it might be worth a shot to access SIMD without machine code.
My NEON is very rusty but I think it could be something like this:
notzed@minized:src$ arm-linux-gnueabihf-objdump -dr neon-mat4.o neon-mat4.o: file format elf32-littlearm Disassembly of section .text: 00000000: 0: f4a02caf vld1.32 {d2[]-d3[]}, [r0] 4: f4210a8f vld1.32 {d0-d1}, [r1] 8: f2000d42 vadd.f32 q0, q0, q1 c: f4020a8f vst1.32 {d0-d1}, [r2] 10: e12fff1e bx lr
As can be seen from the names I started with a "simple" matrix multiply but whittled it down to something I thought the compiler could manage after seeing what it did to it - this is just a meaningless snippet.
After a pretty long day at work I was just half-heartedly poking at filling out the frontend to the epiphany gpu but just got distracted by whining at the compiler again. I should've just started with NEON, after a little poking I remembered how nice it was.
I've spent over 10 hours the last two days trying to solve the same problem. Apparently it is impossible to get rid of tooltips in Win7, which makes it absolutely unusable for me. I've wasted my time. I've wasted my money. Just installed Ubuntu which doesn't cost a penny and with one checkmark I get to disable all tooltips. With Microsoft I get to pay $100 for something that seems intentionally designed for maximum annoyance. Goodbye Microsoft. Last dime you ever got from me.
Had a laugh then thought fucking around in regedit wasn't worth it and so left it at that.
But yeah tooltips suck shit. If your GUI needs tooltips to be usable, it just needs bloody fixing. Icons just don't work when you've got more than a dozen or so choices (not much does for a human).
I had intended to just get a triangle out of an epiphany core, but it just didn't end up happening. I had to muck around getting the rev1 to a 'working' state which took surprisingly long. There are a lot of weird shitty changes to ubuntu (i mean, runlevels, why the fuck would anyone want those???) that took me a while to wade through. I did run the C code I have which outputs to the framebuffer, which worked but is a bit slow. I did seem to have weird issues with USB but a reboot more or less fixed that, with it running through a powered hub. nfs was super-slow till i set it to nfs3 mode. Apparently that's a thing that happens. I also ran one of the ezesdk tests and that worked ... i wasn't sure if that would.
And god, what the fuck have they done to the gtk+ version of emacs? It's like a really ugly version of notepad. I wish I had have found the emacs-lucid packages an hour earlier, would've saved my throat a good deal of violence. But I guess at some point the lucid version wont work anymore, I might have to find another editor (yeah it's that bad).
And what's with all the config/startup shit in debian and ubuntu? Run one tool and it gives a completely obtuse but apparently deep and meaningful message about using another tool, the other tool doesn't know what the fuck's going on and in the end you just edit a file by hand? Why is there even more than one? apt-get/dpkg/whatever else is bad enough. What sort of genious thought that "update-rc.d" was a nice name for a command anyone might ever want to run, ever? Trying to find solutions using a search engine is becoming pointless: it's the blind leading the blind and everything is years out of date. Try finding out how to disable screen blanking on the console for example?
This worked for me:
echo "setterm -blank 0 -powersave off -powerdown 0" >> /etc/rc.local
Net"work"Manager was still spewing pointless shit to the logs whilst trying to "dynamically manage" a fixed ethernet cable ... so fuck that right off. Although i wish more shit actually logged happenings: it seems almost nothing logs when anything goes wrong. I don't see how dedicating a whole virtual terminal to the almost completely information-free "boot.log" is of use to anyone. The packaging system seems to have turned into a sort of enterprise configuration management tool: and you know what, they suck. They're slow and cumbersome and buggy as all shit, and we know because it's linux some doe-eyed fool will come along in a year or two with a new and even more broken 'fix' for all the brokenness.
I can't believe after 20 years of this shit ... it's now way more broken than how it started. At least back then the problems were restricted to hardware support. Now that's fantastic the software has all been fucked up by people poking their noses into places they have no fucking business being.
And i'm still underslept with a headache, with added fun of cold and hungry.
After the last post I kind of remembered one reason to split the work across cores: calculating the reciprocal of 1/w is somewhat expensive unless it can be batched up.
So I was up way too late last night just trying different snippets of code to address that. I think I will go the branchless loop thing that performs the z-buffer test and in-triangle tests separately and then outputs a compact set of coordinates. The compiler was doing some funky stuff but I got some hand-rolled code down to like 10 cycles per pixel (and that could include the 1/w interpolation too); the only real problem with that being the memory required for the output :-/
A separate loop can then just calculate 1/(1/w) to a table (at something like 16 cycles per pixel), and the final loop can then interpolate all the varying values without having to many any decisions about which are live pixels. Without this kind of split there isn't enough registers to keep everything in registers within the inner loops.
Because of the memory it may have to do all this in several batches - slivers of 64 pixels each or somesuch.
But I kinda gave up after today and just worked on a "simple as possible" Java "gpu" to try and have something positive to hang onto after a miserable day (and i started before I got nfs fixed). I needed something which is the distilled/captured knowledge of what I know "so far' as a training simulator. There's still some stuff I need to work out wrt the 3d maths and it's just easier playing with some simple code to do it.
This for example is the code which generates the typical hello world example from OpenGL:
float[] vertices = {
-0.75f, -0.75f, 0, 1,
0.75f, 0, 0, 1,
-0.75f, 0.75f, 0, 1,};
void helloTriangle() {
Viewport vp = new Viewport(0, 0, width, height);
PrimitiveTriangle tt = new PrimitiveTriangle();
tt.setup(vp, 0, vertices);
// red, green, blue
tt.setVarying(0, 1, 0, 0);
tt.setVarying(1, 0, 1, 0);
tt.setVarying(2, 0, 0, 1);
float uniformA = 1.0f;
tt.draw(pbuffer, zbuffer, width, (float[] varying, float[] pixels, int x) -> {
pixels[x + 0] = varying[0];
pixels[x + 1] = varying[1];
pixels[x + 2] = varying[2];
pixels[x + 3] = uniformA;
});
}
This is functionally equivalent to the low-level part of a gpu driver/hardware after the vertex shader (more or less).
Here the lambda expression is the fragment shader. The main problem with using Java as a fragment shader language is how ugly vector/matrix/complex maths ends up being when you need to use flat arrays for efficiency.
Right now this isn't really much more than a training tool and intellectual curiosity, but it's food for thought that computer systems (cpu+gpu+other) and compiler technology is explicitly working toward a point where code such as the above would be the way you "program" graphics drivers. And it would run just as fast as any other driver software. There will probably still be a need for some fixed-function units but these could also be encapsulated as co-processors. The reason this would be possible now when it wasn't previously is due to the technology that makes HSA possible.
A Saturday passes ...
I had a lot of trouble with the matrices and some with the triangle direction - as is usually the case with 3d maths. After playing with some opengl3 tutorials I got it enough worked out to get somewhere. I also played with framebuffer, javafx output, and parallel streams for the tile rendering. Oh and using fixed-point for the triangle edge calculations, which fix some rare edge cases with the edges and might be easier to optimise. And trying to optimise the reciprocal calculation and changing back to the fglrx driver as side-missions (so i could run the gl3 examples - for whatever reason mesa wasn't doing the job, except i forgot which kernel is the one I need to use and the one that mostly works causes some nasty bugs in X). Well you know, a ton of stuff - i lost track of time and suddenly it was 5am.
I should really add some lighting but it's quite mesmerising in full-frame-rate motion all the same. Ok result for a week of late nights piss-farting about.
Still no epiphany code; next perhaps?
Despite any arguments to the contrary it's pretty obvious why ext4 was broken in this way: blah blah ... benchmarks look better ... blah blah.
I just can't believe the distro "community" or any sysadmin actually puts up with this sort of nonsense given there are so many other (excellent, stable) filesystems to choose from. For a so-called meritorious-based "community" this reeks of the same type of following the "industry default" that lead to the disastrous wintel-fucked-over lost-decade of the 90s.
Come on, it's just shit, use something else. There's no excuse. And rather than focusing on benchmarks, isn't it about time the filesystem writers focussed on robustness? I mean come on, why the fuck do I still have to unmount a removable disk before taking it out? That's some other fucked-up-shit that was introduced in the 90s.
I got sick of it so a few weeks ago I changed to developing on the parallella via a nfs disk. Since the TOD clock on my rev0 board is out of whack since changing to the rev1 distro (drifts about 1s per minute) i'm usually editing and compiling on my workstation as well; which makes everything quicker and easier as a bonus. I have a rev1.1 board but haven't tried it yet because the rev0 is working well enough for what i'm doing and my desk is a bit cramped (hmm, but should do a burn-in test soon).
I was looking into a NAS box to just centralise all the "bulk" filesystems of all my computers but couldn't decide on one to buy and then thought that since they were probably running gnu/linux anyway I could at least see if it would be workable (it is). I probably don't even need that magnitude of space anyway: I had another workstation just running to record tv using mythtv for the last few years - but I stopped watching any of it months ago and last time the power went out during a storm I just left it turned off (mysql is another piece of shit so it usually requires some massaging to work after a reboot too, so i saved myself some hassles). It wasn't the original goal/use of the machine it's just that it's in a poor location and then I got other computers.
I did get a usb drive instead, which was probably a bit pointless in hindsight and it's just sitting on the other one I barely use, collecting copious amounts of dust infront of the telly. It seemed like a good idea at the time; I guess i'll get a nas box one day although not for archiving movies or tv series i'm just not going to end up watching.
Commercial TV here is almost unwatchable now with a recording and even with a recording it's a hassle skipping through the ads - there is almost more ads than tv and it's often the same annoying ones over and over again. I'm barely watching the footy this year either - which is normally something to have in the background at least during a wintry weekend: when channel 7's "a-team" are commentating it's just too hard to watch with the sound on at all - fuck fuck bruce macevaney[sic] and all his fucking inane and repetitive one-liners - and i've had more than enough of bunnings "team members" telling me about their shitty cheap imported junk every time a goal gets kicked. And since channel 7 bought the SANFL rights they barely show any games - ABC at least had one match of the week every week, with no fucking ads (actually 7 are showing less national games too, pushing people to their paytv stuff). Most of the rest of the "content" is pretty crap anyway (dumbed down way too much, and/or based on idiotic premises), or repeated to overload. Lately the thing on commercial tv seems to be to show a series as fast as possible by showing 2 or 3 (or more) back to back, once a week until they're all shown. Must be some marketing junk about 'captive' audience but I can't see that working for long if they then repeat the same short series every 6 months and when tv's come with video recorders built in.
I guess on the plus side ... it means i've been doing a lot more hacking.
It's my weekly RDO today, not sure what to do. Should get a quote for solar hot-water or new stove-top or a good number of other things to do around the house but, well, I just don't want to deal with it. Too cold to do much outside - i think the storms have cleared up but a full day of still grey overcast sucks the heat out of the world. Should at least run the vacuum around a bit and load the dishwasher. After that I might do some hacking if i can think of anything interesting to hack on although my brain is still a bit fried from ezetime and work. I've been cranky as hell this week from work and interrupted sleep so maybe I should just do a bit of fuck all.
