This Week in Silver Bullets
Continuing the tradition of the weekly news from the Kubernetes and container industries.
The good news this week is that container prices have hit a five year low. Shipping containers, that is, of course. I don’t need to hit you over the head with the metaphor; plenty of vendors, from Docker on downwards, have been doing that for years.
Michael Winrow writes for the BBC:
In September 2021 the price of a 40ft (12m) container hit a high of almost $10,400 (£9,160), according to the World Container Index produced by analysts Drewry.
Since then the price has fallen back to around $2,773, well below the five-year average of around $3,759 according to the Index.
After the Ever Given incident, Jake Slinn became the face of the container salvage industry, and he is featured again in Winrow’s article. If your container is unusable — say, it’s filled with rotten cabbages — then Slinn’s company JS Global can coordinate its contents being turned into biofuel. In case your container is filled with 20 tons of abandoned breast implants, Slinn can help you out too: they “were shredded and then turned into refuse derived fuel and used to generate electricity.” In an energy crisis, every little helps.
A little more from Detroit
(This section was accidentally removed from last week’s newsletter by gremlins; we apologise for the error.)
KubeCon marketing continues after the conference has ended. Jan Mundin of uptime.build went around KubeCon and took a picture of every booth, building a word cloud of the most commonly used words. “Security” was hot, “automation” was not; “platform” in, “devops” out. Of note, there were only three mentions of “serverless” throughout the show floor.
Of course, sponsoring the conference is not the only way to be present. Early stage startup LeakSignal has built a “MicroWAF for Microservices”: a Wasm filter for Envoy to move Layer 7 traffic analysis from the CDN into the mesh data plane. LeakSignal put all their efforts into the first two days of the conference. They attended EnvoyCon, ServiceMeshCon and Wasm Day, validating their ideas through conversations with engineering leaders in attendance .With marketing restricted to putting flyers on chairs, they were gone before the vendor showcase was even set up!
Are you working on something that we should mention in Let’s Get To The News?
The future is not evenly distributed
Ahead of AWS re:Invent next week, Amazon CTO Werner Vogels has shared Amazon’s Distributed Computing Manifesto, written in 19981. Vogels warns that the ideas in it might sound obvious, but in fairness, that’s because Amazon invented them in 1998, and we’re largely used to their cloud implementations and open source derivatives.
In more recent history, Tim Bray shared a the story of why Amazon decided not to develop a blockchain product, and the story of how a distributed system of a company can ignore that insight and build one anyway.
On deployment
Property rental company Blueground has described how they create deployment previews on Kubernetes using ArgoCD. If you’ve ever submitted a pull request to open source documentation in a project like Kubernetes or Istio, you may have noticed a preview URL added, pointing to a service like Netlify. The Blueground engineering team took this concept a step further, deploying all the services that make up their application, and providing a preview URL for the developer to share internally. These previews run for a limited time, with the ArgoCD ApplicationSet feature allowing them to be cleaned up.
Have you heard of Heroku? Do you think that it would be nice if you could do Heroku-like deployments to Kubernetes? Developer tooling startup Jetpack2, has open sourced their internal tool for such. Launchpad3 is designed to stop developers needing to break out of their flow to think about Kubernetes The announcement was coupled with Mission Control4, a Kubernetes configuration service that Jetpack will offer to provide access provisioning, secrets management, and preview URLs for projects.
Having started a new job at Radboud University, ex-GKE SRE Miek Gieben has found himself having to deploy things to machines that aren’t running Kubernetes. So, he’s started a new project to do GitOps on things that aren’t Kubernetes. This is a very early project, but Miek’s projects have a history of going on to be quite important. I think this is one to keep an eye on.
Kubernetes promises multi-cloud portability, but does it really deliver it? McKinsey Digital did an experiment with the simplest of workloads: Google Cloud’s “Hipster Shop”, which is literally a set of Kubernetes manifests. The consulting team found it effectively took two days to get it deployed on any of GKE, AKS, EKS or ECS. The differences in DNS, certificate management, monitoring and other provider integrations were such that even the most Kubernetes-native of software required thought and consideration when deployed somewhere different than its intended location.
On the importance of a good name
API testing company up9 released “Mizu” earlier in the year. I have a memory of considering it for the news on the old show, not immediately understanding what it did, and ignoring it.
The software has now been relaunched as Kubeshark, which hints at what it actually does: it’s Wireshark for Kubernetes. Initially a side project of up9, Kubeshark has spun out of that company and will be maintained by its creators as an open source project.
The documentation is now much clearer on what it does, and how it works: a DaemonSet on every node allows you to capture traffic from selected pods in pcap format. On one hand, this is very cool: on another, this a reminder that a malicious app can easily tap all traffic in your cluster if it can get the right permissions.
On the importance of a good story
ARMO, the open source security company with excellent taste in employees, has published the results of an survey performed earlier this year. 55% of respondents are using open source tools for Kubernetes security, with almost a quarter using 5 or more distinct tools. Common challenges with open tools are integration, management and setup; common challenges with proprietary tools are their opaque nature and cost. The report says that 95% of people scan their clusters for misconfigurations regularly, which is good, but almost 40% of executives expect that scans happen every few hours — which is interesting.
The Sigstore team has published a paper at the 2022 ACM CCS conference, describing their implementation. The authors proudly comment on how the data in the paper is already out of date, with three times the number of entries in the Rekor supply chain transparency log then when the paper was drafted 9 months ago.
Does the idea of being able to save a copy of a running container sound interesting to you? Aside from bringing us one step closer to the ideal of moving processes from one server to another5, backup/restore and forensic use cases which have driven this feature to Alpha in Kubernetes 1.25 under the name container checkpointing. Martin Heinz has written an in-depth guide to the process, showing you how to enable checkpointing support, then save and restore a container.
Round-up
Sysdig offers some hints on how to secure Helm. (Eduardo Mínguez / Sysdig)
Just because a bug is fixed doesn’t mean it can’t bite: learn how Trendyol found out that cAdvisor was slowing down their disks, even though that behaviour should have been fixed many versions ago. (Muhammet Ozekli / Trendyol)
The Go Readability Team at Google has published the internal style guide that it uses for the Go Programming Language.
2023 is going to be the year of multi-cloud Kubernetes, server-side WebAssembly, and Linux on the Desktop.6 (David Linthicum / InfoWorld, Beth Pariseau / TechTarget)
Emily Trau and Jamie McClymont found some vulnerabilities in Tailscale, which were very quickly and responsibly fixed.
The UK Competition and Markets Authority is investigating Broadcom’s acquisition of VMware. (Paul Sawers / TechCrunch)
The countdown to Kubernetes 1.26 has begun with a blog post on upcoming removals, deprecations and major changes. 1.26 is due out on December 6. (Frederico Muñoz)
Goodbyes
For the past 7 years, a startup called Kite was working on using AI to help developers write code. Now you may think “that actually became a thing quite loudly last year”, and you would be right (or be involved in a class-action lawsuit)7. However, it never became a thing which would 10X developer productivity, and Kite’s 500,000 users, while happy, weren’t willing to pay to use what they had built. Thus, they are now saying farewell. Their have open sourced almost all of their code, with their core repo containing more detail about what they tried as a company.
From the two-sides-to-every-story department - never mind crypto, is the cloud itself having a negative impact on our environment? Steven Gonzalez Monserrate, a “cloud anthropologist” and PhD candidate at MIT, has been conducting ethnographic research in data centers since 2015. His research is summarized in an article at Wired, which suggests that the sky is (quite literally) falling, and we immediately need to break up Big Cloud. Remember kids, not all providers have the same ecological impact, and you can be cloud native without being on a cloud. In an energy crisis, every little helps.
Finally, we pay our respects to Fred Brooks, who passed away this week. Fred was the prototypical “computer architect”, leading the team that built the IBM System/360 and describing stories from the era in the legendary book The Mythical Man-Month. (He also invented the byte. Someone had to.)
I would like to quote from the personal recollections of Steven M. Bellovin, a one-time student of Brooks:
Late one spring, he asked me what my summer plans were. "Well, Dr. Brooks, I’d like to teach."
Brooks demurred: "Prof. X’s project is late and has deliverables due at the end of the summer; I need you to work on it."
"Dr. Brooks, you want to add manpower to a late project?"
He laughed, told me that this was a special case, and that I should do it and in the fall I could have whatever assistantship I wanted. And he was 100% correct: it was a special case, where his adage didn’t apply.
And that’s the news.
The statute of limitations on leaking at Amazon is thus 24 years.
A company that really should have considered if their name might be in use in the open source world already
Docker demonstrated this in 2015, but has scrubbed the video from the internet. We are petitioning them to have this important piece of history reinstated.
And Mastodon, of course.
With exquisite typography!