Graduation Snippets – Docker 1.0, RHEL 7.0, Firefox 30.0

Docker 1.0: The Docker container management platform has hit version 1.0 though the major work had been done by version 0.11 – this is the project’s graduation, acknowledging its ready for production. The actual packaging and management software is going to be referred to as Docker Engine now as the announcement is also the signal for Docker (the company) to roll out 1.0 of Docker Cloud, a platform for sharing Docker packaged apps. Actual changes in 1.0 are things like a new COPY command and an improved ADD command for developers and the ability to pause and resume running containers, added XFS support and performance improvements in container removal. Make a note too. Posts 2375 and 2376 are now officially the HTTP and HTTPS ports for Docker. Docker has changes how people think about package and run applications on Linux and all it would need is for major players to adopt Docker and … oh Google’s added App Engine support for Docker to go with its Compute Engine support and then there’s…

RHEL 7.0: Red Hat has rolled out version 7.0 of its enterprise Linux and 7.0 is looking like a cracking release. Top of their highlights, containers and Docker support, XFS as default file system and new caching file systems (Btrfs is still experimental), systemd and new management components and more capabilities to work with Windows domains. The release notes as with all Red Hat releases are comprehensive and cover things like the switch to GNOME 3 on the desktop (while retaining a classic shell). RHEL 7 is the commercially supported upstream for other distributions, most notably the CentOS distribution which is working on its CentOS 7 release – no dates on that yet but it is the first test of the new relationship between Red Hat and CentOS.

Firefox 30.0: Thirty… As Firefox versioning heads out of the twenties, the [release of Firefox 30.0] has brought sidebar button for toolbar, support for GStreamer 1.0, command-E find selected, various developer changes and five critical and two high security fixes. Now it is thirty, Firefox is well on its way to settling down to a boring life where change is mostly about moving the furniture about and keeping an eye on the neighbours. The place to look for excitement is Mozilla’s Servo browser, being developed in Mozilla’s Rust language and is developing steadily.

Docker 0.9, Vagrant 1.5 and Xen 4.4 – Virtually Snippets

Docker 0.9 unloads: Docker bumps its version number to Docker 0.9 and as it approaches version 1.0 makes a big change. Docker’s been pretty tightly tied to Linux Containers (LXC) technology to run applications packaged with it but in 0.9 there’s now execution drivers so the option to plug in any one of a range of isolation systems is now available. “OpenVZ, systemd-nspawn, libvirt-lxc, libvirt-sandbox, qemu/kvm, BSD Jails, Solaris Zones, and even good old chroot” are on Docker’s planned list with more to come from various projects. There’s also a new libcontainer which lets Docker plug straight into the Linux kernel to control things – this Go library is likely to see a lot of use outside of Docker too as it wraps up container configuration into a neat JSON specified bundle. Next stop for Docker is a production quality 0.10 which will serve as a release candidate for 1.0. Its lively down at the docks.

Vagrant 1.5 roams out: The developer environment manager Vagrant has been updated too. The new Vagrant 1.5 has added a sharing system to make collaboration easier, versioning for boxes, rsync and smb sync’d folders and Hyper-V support. Simpler SSH authentircation setup, a reworked plugin manager and support for Funtoo, NetBSD and TinyCore Linux as guests wrap out the wedge of features in this release. Alongside the release is the announcement of Vagrant Cloud, a hosted box sharing service built to use Vagrant 1.5’s sharing functions.

Xen 4.4 meditates: Meanwhile, the other Linux virtualisation platform, Xen, has made the first release on its aspirational six month cycle (taking 8 months in this case). The announcement for Xen 4.4 highlights an improved libvirt/libxl interface for better integration with VM managers or cloud platforms, a more flexible event channel interface allowing for over tens of thousands of guests and a rapidly maturing ARM port now with a stable ABI going forwards. There’s also a ‘tech preview’ of nested virtualisation on Intel.

LLVM 3.4, Arch 2014-01-05, Mirantis OpenStack 4.0 and Paper encryption – Snippets

• LLVM hits 3.4: The LLVM project’s compilers and more toolchain has reached version 3.4 and the announcement counts down the new features; Clang now has all of the working draft for C++1y standard working, a better static analyser, a “clang-format” for beautiful code in your preferred style and an experimental driver which should let Clang be used with Visual Studio. There’s also lots of performance enhancements in the code generator. Read more in the release notes and if you’re the kind of person who builds their own LLVM kit, head to the releases page to download.

• Arch’s first 2014 update: The first of what will be many, the Arch Linux project has released an update (2014-01-05) to the distro. If you already use Arch, you know that as long as you are up to date you don’t need this. For folks wanting to check out Arch, this update is where you’d start. Well, there and the installation guide or beginner’s guide.

• Mirantis OpenStack update: Mirantis have released Mirantis OpenStack 4.0 which you can download. It includes a number of “hardened” packages and the Fuel management tool which can deploy out to CentOS or Ubuntu.

• Paper powered encryption: The folks at LightBlueTouchPaper have come up with an interesting little paper based, one-time pad driven encryption scheme with a Python script for generating encryption tables. Read more and generate a table or two at the blog posting.

Debian 7.3, Dart at ECMA, Cloud-stealing – Snippets

• Debian 7.3: The latest patch rollup update for Debian 7, Wheezy, has arrived. As usual, don’t throw away your media as all the changes are available as existing updates. This is just rolling up all the updates to date to make new installation media for newcomers. The announcement lists all the changes, security fixes and two packages that were removed. Further info at the release information page for Debian 7. Valve’s SteamOS beta, the operating system for its PC/Console SteamBox, is based on Debian.

• Dart goes to ECMA: Google has made its move to make Dart, its JavaScript replacement language, a standard with the creation of TC52 (Technical Committee) at ECMA. Google says that since Dart 1.0’s release the language is production ready and ready for standardisation. Ready, yes, but it ain’t a standard yet… but people are picking up on it – For example, Notch, creator of Minecraft, used Dart in his Ludum Dare (48 hours to write a game) competition entry – and you could even watch him work in the Dart Editor live.

• Securing clouds: Luke Chadwick had a nasty surprise when his Amazon Web Services bill leapt from $69 to $3000. He’d accidentally pushed his AWS credentials into GitHub and someone had decided to use them to load up 20 large AWS compute engines to do Litecoin mining. As the Hacker News commenters note, that’s $3000 burnt to make a whole $40 of Litecoin; the comments also include suggestions on securing code when checking in by auditing and scanning and how to make the AWS dashboard alert you when something out of the ordinary is happening.

Firefox 26, Netflix’s Suro, Vagrants and Dockers and Websockets for all – Snippets

• Firefox 26 digs in: Today we’ll see the release of Firefox 26, latest in the overly regular Firefox release cycle. From the (currently beta) release notes, we can see the big changes. All but the Flash plug-in are now click-to-play by default, Windows users can update their Firefox without having to write into the Firefox folders, the password manager can handle password fields generated by scripts and on Linux, if the installed gstreamer can handle h264, so can Firefox. A couple of fixes, some developer enhancements and thats about it. There’s also a Firefox for Android update due today. The release notes note some performance improvements, the same password manager enhancement and some fixes. The developer page for Firefox 26 covers changes of interest to developers in more detail. Firefox 26 will be turning up in updates and for download later today.

• Netfix’s Suro goes open: From the people who brought you a cloud full of monkeys… Netflix’s latest open source release is Suro, an application monitoring system used by the video stream vendor to track the behaviour of their Amazon AWS deployed applications. Originally based on Apache Chukwa and adapted to fit Netflix’s demands, Suro pulls the company’s monitoring data from the various app clusters and pushes it to S3 (for Hadoop based analytics), to Apache Kafka (and on to Storm, Amazon ElasticSearch and Druid and to other event processors. There’s a lot more detail in the announcement including in production stats and how the pipeline is used to analyse errors.

• Vagrant meets Docker: The latest update to Vagrant, version 1.4 has been announced and the big improvement in system that has traditionally been used to create automatically reproducible development environment is the addition of Docker support. The Docker provisioner can install Docker and then lets Vagrant cirtual machine pull and configure Docker containers within it. There’s also some enhancements to the scriptability of Vagrant itself, the ability to require a particular version of Vagrant and support for standalone file sync plugins.

• websocketd: And finally, have you wanted to make a shell script or other app into a WebSocket server but lacked a library or access to the code to do it? Websocketd might be the answer as it turns anything with console I/O into a WebSocket server in a style rather reminiscent of CGI. Remember, most command line applications are not suitable for being exposed to the raw web, but the app could get you out of a hole when prototyping.

And, for reference, everything mentioned today is open source software.

Multiprocess Firefox, Kexec and Secure Boot, Poisoning GCC and OpenNebula 4.4 – Snippets

• Firefox goes multiprocess: Some years back, Mozilla embarked on the Electrolysis project to give Firefox a multiprocess architecture, where each web page ran in its own process. This idea isolates web pages from crashing each other and should have performance benefits too; Google’s Chrome, for example, was built with such an architecture. Unfortunately, a year later Mozilla put that effort on hold to work on things which would give quicker returns. Well, now it’s 2013 and the project in back and already in the nightlies. A full write up on Multiprocess Firefox is available in Bill McCloskey’s blog which explains there’s no release date for this work yet, how to enable it if you want to try it out and how things will break and how add-ons are affected.

• Kexec and Secure Boot: Matthew Garrett has written up why kexec is disabled in Fedora when booted with Secure Boot enabled. Worth a read as it shows why being able to swap kernels in such an environment is a bad thing.

• Poison for GCC: One thing Microsoft have done well is providing red lights for dangerous function calls (like strcpy and sprintf) in their tools (by adding a header file banned.h). Now, Leaf Security Research are creating a version for GCC with a Github project to create a “gcc-poison.h” file. Using it could help developers find those nasty vulnerable, error-prone functions hidden in their code base.

• OpenNebula 4.4 goes “Retina”: The other other open source cloud platform, OpenNebula, has just been updated to version 4.4, codenamed Retina (after the Retina Nebula – this project has the best codenames). The update supports multiple datastores with scheduling policies to spread loads across different VMs and their associated storage. For more details, check the release notes.

Docker for all Linux distros, DPorts and more for DragonFlyBSD and advice for coders – Snippets

• Docker 0.7 unloading: With Docker 0.7, the Docker developers have made a big leap in Linux coverage. (If you are new to Docker, read the introduction to it I did for the Linux Foundation). Under the covers, Docker has used storage drivers to maintain images on disk, but up till now they’d needed a patched Linux kernel for that to work. A patch from Red Hat has changed that though and adds “DEVICEMAPPER”, a storage driver which used copy-on-write LVM snapshots and doesn’t need a patched kernel, to the list of storage drivers. The selection of the driver needed is done automagically and the resultant images are interchangable between different drivers so there’s no driver lock-in. That all means that Docker now runs on Fedora, RHEL, Ubuntu, Debian, SUSE, Arch, Gentoo and others. More drivers are coming too, for BtrFS, ZFS, Gluster and Ceph. Other additions, merged in the 0.6 cycle include offline image transfer, better port redirection, linkable containers and descriptive names for containers.

• DragonFlyBSD updated: Version 3.6 of DragonFlyBSD – the now ten year old BSD project that sets out to give BSD native optimised clustering capabilities – has been released. The update standardises on Dports and pkg for installation tools, making around 20,000 packages available, and the process of building those 20,000 packages in parallel has allowed for the testing and near elimination of kernel contention with more cores scaling up the improvements made. There’s also i915 and KMS support, albeit experimental, and updated localisation. DragonFlyBSD is still using its HAMMER filesystem with work on HAMMER2 carrying on into DragonFlyBSD 3.7.

• Coding Advice: Whether your learning or experienced, this article offers sage advice on how to approach coding. While we’re on the subject of advice, here’s some false things that programmers believe are true about geography, addresses, names and time.

Hey! Presto – Facebook’s latest open source code

Facebook, in their now traditional goal of taking on big data problems, solving them and then open sourcing the result, have open-sourced Presto, a distributed SQL query engine “optimized for ad-hoc analysis at interactive speed”. This type of app is designed for the folks who need to work out what people who like chips and cheese and rock but dont like bagels or opera also have, statistically, in common. Its a simple enough question, but when you get up to Facebook scale, its a hard question to answer. This is the land of Hadoop and Hadoop has its own SQL-like query engine, Hive. 

But unlike Hive which converts queries into MapReduce tasks saving intermediate results to disk, Presto has a query and execution engine which runs in memory and is pipelined through the network. Presto is implemented in Java for easy integration with other parts of Facebook that are also built in Java and compiles parts of queries down to bytecode, letting the JVM JIT compile to machine code to get the best out of the Java environment. Although it doesn’t need Hive, Presto does need a datasource for its queries and it includes a plugin for Hive, though it only uses the Hive metastore service, presumably to obtain structural information, and then accesses the data over HDFS.

The Facebook announcement says “Presto is 10x better than Hive/MapReduce in terms of CPU efficiency and latency for most queries at Facebook” and has been in use internally since Spring of this year with multiple deployments and one cluster scaled to a thousand nodes. A thousand users actively use it with 30000 queries and processing a petabyte a day. Thats a good work out for any big data offering.

There’s plenty missing from Presto; various joins and aggregations are restricted and there’s no way to write results back into tables – they go straight to the client. Those issues, plus improved performance, query accelerators, hot cached data subsets and a high performance HBase connector are all on the roadmap for Presto.

Presto is licensed under the Apache License 2.0 but does not appear to be heading to the foundation with active development taking place around Facebook’s GitHub repository.

EOL for Python 2.6, Docker Inc and more iconic fonts – Snippets

• Python 2.6 signs out: Python 2.6.9 is the last source-only security fix release for the Python 2.6 family. The 2.6.9 release sees 2.6 officially retired after five years in the field. If you are still running 2.6, UPDATE! At the other end of the scale, Python 3.3.3 got its first release candidate with full support for Mac OS X 10.9 Mavericks.

• dotCloud becomes Docker Inc: Acknowledging how important its Docker container software has become, dotCloud has announced it is becoming Docker Inc. The platform-as-a-service business of dotCloud will be maintained, but the company’s resources are going into Docker, Docker services and building out the Docker ecosystem.

• More icon fontage: Bootstrap is not alone in having a fine icon font for its graphical imagery. Say hi to Ionicons, created for the Ionic front-end framework. Very stylish, and MIT licensed open source.

MongoHQ’s security breach holes others

If you were using MongoHQ‘s SSD backed MongoDB hosting, be prepared for them to be in touch as they’ve been at the sharp end of a security breach. But it’s not just direct users of MongoHQ’s services that should be concerned – users of services which make use of MongoHQ need to put on their worrying hat too. For example, MongoHQ hosted Buffer‘s databases and that has been cited as the cause of the  social media connector’s security breach. Another company, cloud based continuous integration specialists CircleCI, has also been compromised and issued its own security advice (through a statuspage.io supplied status page which as I write, has fallen over). They probably won’t be the only ones either.

With an interconnected set of reliant services, the services at the bottom of the stack are often the ones which have the biggest target on them. To draw a parallel, if you want to make the Jenga stack fall over going for the bricks at the bottom is a good strategy. Hitting popular data-service providers in the cloud pays big for an attacker; an original target may come with many bonus victims and the ripple out of awareness of the compromise can provide a bigger window for the attack to fill its swag bag and make out through the window. Which is why, when you are looking at a service provider in the cloud, you need to make sure they have good defences, an effective monitoring system and a notification system which lets clients react quickly… and that’s not a “service status page which updates regularly”. It’s the same list you should have for your in-house and condensate* systems too.

* systems that use cloud technology but aren’t actually up in the cloud.