Elsewhere: NewsBits – Redis 5 RC 5

I’ve been doing my usual Friday news gathering for the day job and that means here is todays NewsBits…. Here’s what’s in it:

• Redis 5.0 gets a new release candidate and controversy.
• Updates for older MongoDB versions.
• A guide to analyzing slow MongoDB queries.
• Making MySQL‘s shell shine.
• Google open up Dataset Search.
• Firefox 62 lands, as does the new ESR release.
• HTTP2 support no longer experimental in Node 10.10.
• VS Code gets a new Settings UI.
• Checkout pull requests with the latest Atom.
• Where to get Java support in the future?
• And whats it like migrating to Java 11?
• And finally an SQL puzzler…

Click here to read Compose’s NewsBits (be meeeee!) for this week

Developer Catchup: Rust 1.0 and Node reunification

Rust wakes up

First up, Rust has reached version 1.0, though this is an announcement that was hardly unexpected. It has a lot to live up to given the Rust web site goes for such unloaded language as “blazingly fast, prevents nearly all segfaults, and guarantees thread safety”. The real test for Rust, at least for me, is how well Servo, Mozilla’s browser written in Rust and the application Rust was created with in mind. It seems this is the best possible test case, so…

There’s already a minimised ARM port which looks to bring Rust’s safety features to RTOS/embedded environments and I’ve come across some systems programmers who are interested in Rust, but not noted much momentum. Rust is living in the higher 50 (51-100) of the Tiobe index which is a very approximate guesstimate at momentum, but better than nothing. What this says to me is that despite arriving at 1.0 complete with packaging system and more, Rust is going to have a long journey ahead of it.

Node reunifies

Back in December, when Node.js and io.js forked, I expressed the hope that it would be a positive fork. Well, now that fork is coming to an end with the reunification, under the umbrella of the Node Foundation. Except there won’t be any merging of code and the io.js repository is being turned into the node.js repository.

Io.js folks will join the Node Foundations technical committee and, going forward, the next Node.js will be based on Io.js code. It’s well done to Io.js for taking action and making good practical and solid engineering steps that made it practically a no-brainer to take Node.js forward. We don’t seem to be completely done yet. The structure for future releases and development still needs nailing down.

So here’s looking forward to the next two releases of Node.js… the one which brings us all Io.js’s improvements like an up-to-date V8 JavaScript engines, and the one after which will probably come with more detail on how the desire for a faster development cycle and the need for stable long lived versions will be sorted out.

If you are looking to get your head around ES6, the next and arriving generation of JavaScript, try out Understanding ECMAScript 6, which is a CC-NC book via LeanPub – currently 30% complete and already full of useful details.

Developer Catchup: New Node, Profanity, Oh-My-Git, Knightmares, Bad Docs and Go Tracing

Node.js 0.12 has arrived with many long gestating changes now available. Io.js has a lot of these in already and a more up to date V8 engine for JavaScript, but if you’re sticking with Node.js releases, this is the biggy. Better more sensible streams, more HTTP sockets and keepalive, a new round robin clustering system and initial support for ECMAScript internationalisation. No, don’t go flipping your production system over to this right now, but do give it a go on your test/staging systems… it’s the future y’know.

Profanity is a console-based take on XMPP messaging, bridging the gap between IRC and desktop clients. Check it out if you live in terminal windows.

Oh-my-git implements one feature of Oh-my-zsh which people really like without needing to switch to Oh-my-zsh or even zsh. It’s a prompt engine which makes your zsh or bash prompt you with details of your current directory in terms of its git status. If it isn’t a git directory, it gives you a normal prompt. Nice work.

Want to hear a horror story? Read Knightmare: A DevOps Cautionary Tale and wince at the pain that took a company down in 45 minutes. Yeah, dodgy dossier timing.

Embedded code run in documentation? Sounds like nothing could go wrong with that… much… well a lot. Well, this is awkward…

First time using Go? Fogleman knocks it out of the park with his pt project, a path tracer in Go which comes complete with examples. He’s got lots of plans for it too so if rendering 3D things is your thing, you may want to check it out.

Developer Catchup: Synchronous Node, Serviced Polyfills, Sparks Sparked, Tangrams Mapped and SHAaaaaaa!

Node.js synchronously: Node.js is sweet if you can adapt to the asynchronous model of start thing, say what you want to do when its done, do everything else anyway. Good for web request handling but bleh for trying to emulate a shellscript. Turns out that in Node.js 0.12 (coming soon? anyone? Bueller?) we get synchronous child processes to now you can run that curl or find or whatever and just wait till its returned with its results. The folks at Strongloop have written about these synchronous child process methods and how they make writing command line utilities in Node easier. Check it out Noders.

Serviced Polyfills: Polyfills fill gaps in browser functionality and standards compliance. The older the browser, the more Polyfill you need to fill the gaps and the newer, the less. But it gets hard working out how much Polyfill you are going to need. Fear not, as Samuel Giles at FT Labs has an answer, “Polyfills as a Service“. Add a simple script tag pointing at a source from the polyfill.io content delivery network to your pages and whatever browser views your page, it gets the polyfill it needs. This is because the system sniffs the browser agent and works out the best set of polyfill based on that. Neat idea, potentially very handy – and you can run your own private version if you need to.

Spark sparks: Apache Spark just got a 1.1 release. Spark is Hadoop data processing engine which can run on YARN-based Hadoop clusters or in standalone mode. Spark 1.1 improves the performance (and they already say they are up to 100 times faster than Hadoop MapReduce) and has SQL layer enhancements. 1.1 also adds more statistical functions, can take steaming data fromAmazon Kinesis and pull data from Apache Flume and more. If your into clusters and data crunching and haven’t looked at Spark, you might want to look into it.

Tangram Mapping: Do you want to render cool 2D and 3D maps? Check out Tangram, a Mapping Library then as it is building out from a WebGL implementation to other OpenGL platforms to make oodly cool dynamic map renders. Very slick.

SHAaaaaaa!: We mention the Google sunsetting of SHA-1 the other week. If you were unsure why this was important, can we send you off to Why Google is Hurrying the Web to Kill SHA-1 which explains why it all and includes a brief history of collision attacks in the wild.

Just released: Socket.IO 1.0, Git 2.0 and OrientDB 1.7 – Snippets

Socket.IO 1.0: Socket.IO has hit version 1.0 – the Node.js and browser library which started life as an implementation of the WebSockets interface and has gone on to “become the EventEmitter of the web”. The 1.0 release and changes are broken down in a blog posting, the first on a newly redesigned, and much more useful, Socket.IO website. In brief, modularisation, tighter code, binary support (so you can emit blobs and buffers), automated testing, better scalability using redis, more integration (including PHP support), better debugging support (and silence by default), sleeker APIs and CDN delivery. And the future plans include handling Node.js streams, Socket.IO support in Web Inspector and Firefox Dev Tools and more language and framework support. A splendid tool to have in your arsenal.

Git 2.0: The distributed version control system which made distributed version control systems cool before even version control could be cool, Git, has reached version 2.0. In the announcement of the new release, there’s a long list of all the changes and notes on backward compatibility. The 2.0 release has been anticipated by the developers for a while so a lot of ground work had already been done in previous 1.x versions making the 2.0 release look more like a minor release than a major version bump but there’s still plenty of changes and a foundation prepared for future changes. On that subject, there’s a promise of a shorter release cycle for the next release as delays have meant a number of features ‘cooking’ for longer in the ‘next’ branch.

OrientDB 1.7: Version 1.7 of the Document/Graph/Sql/NoSQL database OrientDB is available. The announcement for 1.7 notes better perforamnce, new clustering options, support for SSL and sharding, simplified configuration, new SQL commands including parallel queries, plugins for Lucene-based full text searching and more. There’s an Apache 2 licensed community edition of the database and commercially sold and supported professional and enterprise editions.

GNOME, Systemd, Node Packages and a comment – Snippets

GNOME 3.12: And so GNOME 3.12 has arrived with updates all around. Over here at Codescaling, we’re looking forward to the improvements to HiDPI display support . We had been spoiled by Apple’s fairly smooth switch to HiDPI support but on Linux and Windows its all, surprisingly, a work in progress. Our GNOME desktop on a HiDPI dispay brings a range of font sizes to the screen, only some of which are appropriate. (Google Chrome has its own special mix of tiny tabs, chunky (too chunky) bookmark bar and microscopic (mostly) main window so its not just GNOME that has issues). Thats why we’re using HiDPI as our new proxy for up-to-date-ness. Back at GNOME 3.12, there’s also some nice additions like the popover menus for a flatter UI and more work on making Applications more managable entities. Unfortunately, all this won’t be making it into Ubuntu GNOME 14.04 LTS because it feature froze just over a month ago in what could be described as un-cadence. For the curious, there’s already a FreeBSD with GNOME 3.12 VM available. There is also a more traditional Fedora based GNOME 3.12 live image available to download as an ISO.

Systemd 212: An update of bugfixes for the future dominant ‘init’ system in systemd 212. My favourite feature is at the top of the list… make sure that the screen has minimal brightness when restoring settings to stop people rebooting into a black screen. There’s also been work done on timers including the ability to persist timers to disk so they can be used to start processes over a longer span of time and through reboots.

Comment: Steady improvements with GNOME, systemd and Wayland make me hopeful that there will emerge a good end user Linux distro with a solid desktop designed for a general population but configurable for power users – GNOME 3 is well on the way there, but there’s no distro that seems ready to host such a combo as its primary environment beyond Fedora and that’s not aimed at the general populous. Agree/Disagree? Why not try our comments section below?

Node package popularity: It’s the little things that count, and one little enhancement just made to the Node package manager site sets out to make package searching easier by ordering results by popularity – though on testing here at Codescaling, the algorithm will need a bit more work as it seems packages with no ratings and no downloads rise to the top of the list. Still, its good to see work being done on the NPM site, that isn’t security fixes, after the somewhat contraversial creation of NPM Inc . The backend was already getting attention

Node-RED updated, Hadoop 2.3.0 out, NetBeans 8.0 RCs and Skrollr scrolls – Snippets

Node-RED updated: The most excellent graphical UI for connecting the Internet of Things (or just things in general), Node-RED has been updated to version 0.6. The announcement notes the process of separating the admin and server authentication to make deployment more robust has begun. Node-RED has nodes that accept HTTP connections and has a HTTP admin front end and previously these were all under one HTTP authentication mechanism – now the UI and nodes are more separate with the option to set a user/password for each. There’s some UI changes like a search filter for the palette of available nodes and easier flow importing by just dragging and dropping JSON onto the UI. In the node-red-nodes library, they’ve added Postgres, Amazon DynamoDB and Emoncms for more connections. There’s also fixes for the MQTT keepalive handing, an added socket timeout settings for TCP sockets and support for all 17 pins of WiringPi. More generally, there’s a range generating node now and the inject node can send empty payloads if needed. Finally, the MongoDB node now can send a user name and password – something I found I needed when writing this for MongoHQ.

Hadoop 2.3.0 released: In case you missed it, version 2.3.0 of the Apache Hadoop project got a release. The release notes list all the details. The short version is this is mostly about HDFS, the distributed file system and the changes include the ability to class the storage under HDFS so you can make tradeoffs between say spinning media, SSDs and memory, an ability to explicitly cache files or directories under HDFS (and local zero-copy reading from the cache) and the use of HDFS and YARN to simplify deploying MapReduce code. Hortonworks has a good writeup which also looks forward to Hadoop 2.4.0 with HDFS ACLs and rolling upgrades.

NetBeans 8 gets an RC: The NetBeans IDE has hit release candidate for 8.0. This is the version that will include JDK 8 support in the editor, Java SE Embedded and Java ME Embedded support, PrimeFaces code generators, AngularJS navigation and code completion, PHP 5.5 support and much more. There’s a summary in the announcement, a lot more detail in the New and Noteworthy wiki page and a pencilled in release date of mid-April.

Skrollr scrolls in: Recently spotted – Skrollr, a compact parallax scrolling and scrolling animation library for all your Webtml5.0 styled sites including the ability to “scale, skew and rotate the sh** out of any element”.

Facebook’s Conceal, Callback hell and a listening Pi – Snippets

Facebook’s Conceal revealed: Facebook have open sourced Conceal, a library for encrypting files on Android devices. The company uses the library for encrypting data that its apps store on SD cards. It uses pre-selected OpenSSL algorithms, picked for efficient memory management and speed, and gets the library down to 85KB by not trying to be a general purpose crypto kit. An interesting bit of pragmatism which means Facebook’s apps can happily encrypt on low-end Android devices, Conceal is available under a BSD licence with its source on GitHub.

Callback hell: Callbacks in Node.js can get pretty gnarly if you do everything with inline anonymous functions. This blog posting from Strongloop is a handy summary of some of the ways, from nesting, modularisation, async, promises and (soon to come to Node) ES6 generators. So callback, much techniques.

A Pi that listens: Meanwhile, a nice little Instructable covers converting an old bakelite Televox intercom into a voice controlled personal assistant by popping a Raspberry Pi inside.. and a sound board… and some software of course… It’ll probably be quite hard to find another fine bakelite intercom, but the rest of the projects a good starting point for assembling your own style of smart box…

Arduino/Raspberry Pi UNITE!, Node’s power ups and web video’s next battle – Snippets

• Arduino and Pi Together At Last: The folks behind the fabulous BrickPi project have just kicked off a kickstarter for their next project, Arduberry. This time, they are making a board for the Raspberry Pi which gives the Pi compatibility with the world of Arduino shields. Actually, it adds a whole Arduino Uno to the Raspberry Pi and its going to open up a whole world of possibilities… board adapter, IO controller, compact teaching platform. It’s an open design and the kickstarter’s already funded but Pi owners everywhere will want to add one of these beasts; I’ve already backed the project and can’t wait to play when they deliver in (hopefully) May.

• Coming in Node 0.12: Badly titled, but worth reading What’s New in Node.js 0.12 is a posting on the StrongLoop blog about what to look forward to when Node 0.12 arrives. As they point out, its been cooking for nine months but has speed ups for writable streams, TLS support, crypto algorithms, a less strained garbage collector, better cluster work allocation and improved timers. They also have a 15 minute video on more of whats going to land in 0.12.

• Next-gen video royalties announced: The issue of royalties around h264 were core to the resistance to the standard that in turn led to the creation of VP8 and a stall in web video development while battle lines were drawn. Well, that battle is about to begin again with HEVC, the follow up standard to h264, but streamingmedia.com brings news of what those royalties will probably be. As the article says, the one to watch in this game is Google, who despite developing VP8 (and VP9, their open source HEVC competitor), sat on the pragmatic fence over h264, supporting both in a move which left the idealists at Mozilla out on a limb.

Linux 3.13 lands, Node-RED re-flows, FreeBSD 10 close, BBB-SDR challenge – Snippets

• Linux 3.13 lands: It took a couple of weeks longer than expected, but Linux 3.13 has been announced. KernelNewbies has their 3.13 summary up, or if you speak German, Thorsten Leemhuis has his What’s New up on heise. Highlights? New firewall packet subsystem nftables, new multi-queue I/O that works better with fast devices, open source driver Southern Islands Radeon (HD7550-HD7590) 3D support and power management (and HDMI audio) are now on by default. Throw in improved NUMA and hugepage support, squashfs performance and many more additions and changes and you got yourself a 3.13. Look for it to percolate out through the distros over the coming months.

• Node-RED self-flows: I’ve talked about Node-RED before – it’s the neat GUI for connecting the Internet of Things, and more, together. In a neat article, Andy Stanford-Clark looks at making Node-RED make its own flows. Flows are what you create in Node-RED to implement how data flows around and the idea that Andy looks at is making a flow which would squirt out JSON which could be sent to other Node-RED instances to configure them. Imagine, stepping back for a moment, that you could have a flow which looked up all the devices that ran Node-RED and the other devices that could be accessed and created flows for all of them so they could all work together … thats where this idea is going. Worth a read (and if you haven’t played with Node-RED, make some time for it).

• FreeBSD 10 very very close: It looks like FreeBSD 10 is just about to be announced – ISO images have appeared but there’s no announcement quite yet. There’s a frisson of activity around a Hacker News posting that the files have turned up, but it ain’t announced until the PGP-signed fat lady sings.

• BeagleBone Black Radio challenge: The folks at Element 14 are running a BeagleBone Black Radio Challenge looking for people to propose how they’d make best road-test use of a package comprising of a BBB, a 4.3″ touchscreen cape and an AdaFruit Software-Defined-Radio USB Stick. So are you up to test your maker-mettle?