Off-by-none: Issue #40

June 4, 2019

Did your RDBMS just become serverless?

Welcome to Issue #40 of Off-by-none. Thank you for being here! 👋

Last week we reviewed a nifty little update to AWS Step Functions and shared some interesting serverless stories. This week, AWS GA’d an exciting new “serverlessish” way to access your MySQL databases, plus I share a new Node.js library that’ll make using this new innovation even easier. And as always, we share some amazing content from the serverless community.

It was another busy week, so let’s get to all the good serverless stuff! 🎁

When you really, really need to use a MySQL database with your serverless app… 🙈

This past week, AWS announced the General Availability of the Aurora Serverless Data API. In short, it lets you “issue SQL commands against an Aurora Serverless cluster by simply invoking an API endpoint provided by the Data API and the Data API takes care of managing database connections and returning data.” This is a pretty big deal, because as we know, managing MySQL connections with Lambda is a challenge (if only someone had written a package to help with that 😜).

The preview of Data API launched just before re:Invent 2018, to quite a bit of fanfare. Unfortunately, the performance wasn’t great. In November of last year, I wrote a post outlining my initial (poor) experience with it. However, this new release is much better, so I’ve update my post (Aurora Serverless Data API: An (updated) First Look) with lots of new details. Jeff Barr also has an official AWS post that shows you how to set up the Data API and gives some basic usage examples as well.

But wait! There’s more! When I was playing around with the updated Data API, I found myself writing a few wrappers to handle parameter formation, transaction management, and response formatting. I ended up getting just a bit carried away, and turned it into a full-blown, “DocumentClient”-esque library! 🎉 I call it the “Data API Client“, and it’s available now on GitHub and NPM. Your feedback and contributions would be greatly appreciated, and hopefully you’ll find it super familiar and useful for your projects.

Serverless News & Product Announcements 📢

Palo Alto Networks buys Twistlock, PureSec for container security
Wow, good for Ory and the team over at Puresec. They built an amazing product and Palo Alto Networks should make a great home for them. If you’re wondering what’ll happen to Puresec, read this open letter to PureSec’s clients, partners, and friends.

Visual Studio Code · Stackery Documentation
This is simply amazing. Stackery’s visual builder tool now runs locally as a VSCode extension. I’ve played around with this, and the workflow is excellent.

Lumigo adds Java Tracing support for AWS Lambda
Good news for people who enjoy painfully slow Lambda start up times. Hopefully this new feature from Lumigo will help them make the decision to choose a different runtime. Just sayin’. 😏

Announcing Manual Instrumentation for .NET Agent
You can now add manual instrumentation with the new OpenTracing compatible API. According to the post, automated instrumentation support for .NET is on the way.

Just Write Code: Improving Developer Experience for Cloudflare Workers
Cloudflare Workers gets a CLI, new and improved docs, multiple scripts for everyone, the ability to run applications on workers.dev without bringing your own domain, plus a free tier for developers to try it out. I’ve been very impressed lately by Cloudflare’s push to bring computing to the edge. Very cool stuff.

Serverless Developer Acceleration at Trek10
I came across this page on the Trek10 site and thought it was worth sharing. Not only are Forrest and Jared two of the most knowledgable serverless folks I know, they are also huge contributors to the serverless community. If I had a group of developers that needed to be trained to do serverless the right way, these would be the first people I’d call.

Serverless Stories 📖

CI/CD, AWS, and Serverless: 5 tips I learned the hard way
Speaking of Forrest Brazeal, he has a great piece that outlines some best practices for testing and deploying your serverless applications; picked up from his time in the software trenches. 👨‍🏭

How I Built This: IsraelVC
Elie Steinbock built a side project that catalogues the latest investments in Israeli high tech. It uses Google Sheets as a database, but for his use case, it’s probably a fine choice.

No More Marking is going Serverless!
I love reading success stories like this. Chris Wheadon’s short piece explains why they went serverless, and their plans for the future.

Serverless Use Cases 🗺

Sequence Numbering in Serverless via API Gateway
Love this use case, because it’s probably needed by most of us. Sheen Brisals from the Lego Group shows us how to use DynamoDB as an atomic counter for generating sequence numbers. He even shows you how to create an API Gateway with a (😱) Service Integration to use it! No Lambda function required.

Using the MailChimp API with Netlify Serverless Functions
Using a serverless function to subscribe someone to a mailing list is a nice, simple use case. The Raymond Camden shows us how he built one with a Netlify function.

Scheduling irregular AWS Lambda executions through DynamoDB TTL attributes
Good post by Michael Bahr that shows the use of DynamoDB TTLs as a way to schedule events. As Yan Cui has pointed out, the TTLs are not very accurate, so don’t use this if your requirements require precision. Also, Michael doesn’t mention Step Functions as a way to schedule one-off executions without needing a CloudWatch rule.

Building A ‘Serverless’ Chrome Extension
Serverless as a backend for your browser extensions. Bilal Tahir outlines some experiments he did with Google Cloud Functions and how he tied that into his Chrome Extension.

Realtime Object Detection API using Amazon SageMaker and Amazon API Gateway
Great post by Yi Ai that shows us how to take advantage of AWS’s machine learning services to build some really interesting applications.

Running Arbitrary Executables in AWS Lambda — Encrypting a PDF
If you have more complex executable requirements, it’s likely still possible to run them in a Lambda, you just need the right build environment. Keith Coughtrey shows you how to do it by building it on an EC2 instance first.

Easily query AWS service logs using Amazon Athena
Damon Cortesi shows you some need tricks to query your log files from S3 using Amazon Athena. He also introduces AGSlogger, an open-source Python tool for converting popular log formats into columnar-based Parquet files.

Some things to think about when you’re going serverless… 🤔

Thirty serverless architectures in 30 minutes
If you’re still wondering what you can do with serverless, check out Chris Munns’ slides from AWS Summit Chicago. This should get the wheels spinning (or your brain smoking).

Deconstructing Serverless Computing Part 3: Ninety-Nine Platforms but How to Choose One?
Lucian Toader outlines nine different FaaS platforms, including their general and differentiating features. Lots of really great info in here.

How to Save Hundreds of Hours on Lambda Debugging
Renato Byrro outlines some limitations with CloudWatch Logs and how tracking down errors isn’t quite as straightforward as you might think. There are better ways to do this, whether by using Dashbird, or one of several other excellent observability tools for serverless.

Don’t Be Surprised by Your Serverless Bill
Speaking of observability, Yan Cui outlines a few common situations that can result in higher serverless costs. Good news, most observability platforms, like Epsagon, are able to help you monitor these.

Part 1 — Getting started with AWS Lambda Layers
Lambda Layers are a great way to manage shared dependencies across functions. Dhaval Nagar runs through the basics of creating and using one.

Serverless Tutorials 🏗

Serverless Framework Courses & Tutorials
This is a great resource. The Serverless Framework has put together a compendium of courses that’ll help you learn the ins and outs of building a serverless application with their framework.

Getting started with serverless
And if you’re looking for some more serverless courses, this post outlines a new AWS Fundamentals: Building Serverless Applications Coursera course that is now available.

Serverless Gatsby Pipeline with AWS Amplify
A step-by-step guide for setting up and deploying your React Gatsby project using an automated pipeline to a serverless S3 bucket backed by Cloudfront.

Create your first Serverless workflow with Durable functions
Chris Noring gives you the what, why, and how-to, for building with Azure’s Durable functions. Lots of interesting stuff in here.

Ordered queue processing in Azure Functions with Sessions
Jeff Hollan writes about one of his favorite topics, processing events in order (his words). But seriously, using queues that “lock” a session to a consumer to guarantee ordering might be genius.

Serverless Security 🔒

Council Post: Servers To Serverless: Does New Technology Mean New Security
I certainly don’t want to downplay security, but I feel like many of the suggestions in here don’t apply if you’re not managing the containers (read Kubernetes) that run your serverless functions. AWS is really good at security, so I’m going to defer to them for most of this stuff.

10 Things To Know About Palo Alto Networks Buying Twistlock, PureSec
Not a lot of serverless talk in here, but I included it because of point #5: Buying PureSec Is A Meaningful Milestone For The Serverless Movement. I wholeheartedly agree. 😀

Serverless Security AntiPatterns: 7 Serverless Computing Security Mistakes
I certainly don’t want to overplay security (see what I did there?), but I think these tips by Twain Taylor are a good starting point for creating and maintaining a good serverless security posture.

Serverless Reads 🤓

On SQS
Excellent post by Tim Bray arguing that “if you’re building a moderately complex piece of software that needs to integrate heterogeneous microservices and deal with variable sometimes-high request loads, then if your design doesn’t have a queuing component, quite possibly you’re Doing It Wrong.”

Cloud Irregular: Does Anyone Know How Your System Works?
Another installment of Forrest Brazeal’s Cloud Irregular newsletter. His analysis of the problem is spot on, but the solution is hard (and often impossible) for most teams to swallow.

Cost Analysis: Serverless scheduling of irregular invocations
Michael Bahr does some math and details the cost of implementing his serverless irregular scheduler. Spoiler alert, it doesn’t cost much!

Why I believe in the brave new world of serverless
I really enjoyed this post by Zan Markan. It’s short, but it captures and explains the true power of serverless. Lowering the barrier to entry has both its pros and cons, but ultimately it creates a huge marketplace for ideas and innovation. Personally, I think this new world is going to be a good one.

An Intro to the Power of Serverless with AWS Amplify
Jared Short shows you how powerful Amplify and AppSync are by building an old-school “visitor counter” badge.

Thoughts from Twitter… 🐦

Trying to convince security that we should have an SCP in prod with “Deny”: [“ec2:*”, “ecs:*”]. You know, just to be safe. #serverless ~Ben Kehoe
Forget about being safe, maybe just issue this as a challenge for your developers to build things without needing to provision servers.

If you were going to make a falsifiable claim about k8s and serverless adoption over the next 12-24 months, to explain more people getting frustrated with the former and going to the latter, what would it be? ~ Joe Emison
Interesting conversation going on here. People love containers (I mean, hey, they are pretty darn cool), but after having used serverless for several years, I have zero interesting in adopting Kubernetes to solve a problem that no longer exists for me.

How do you install kubernetes on serverless? ~ Aaron Patterson
I had to include this. Some of my favorite replies are “Machine learning to generate blockchain-authenticated YAML?” and “May the pods be ever be in your favor.”

Question: What’s your favourite AWS Lambda event trigger? (yes it’s an odd question, but I’m interested to know) ~ Paul Johnston
Maybe an odd question, but the responses and discussions it started are quite interesting. Worth a quick skim at the very least.

When you’re looking for a more audio-visual experience… 🎧

Screaming in the Cloud – Episode 62: Serverless Storytelling with Anna Spysz
Corey interviews Anna Spysz from Stackery about her unusual path into programming, writing good documentation, and developing serverless applications locally with Stackery.

Investing in Serverless: Less Servers, More Code
Learn about the Azure serverless platform and how Azure Functions, Event Grid, and Logic Apps work together to deliver cloud native experiences.

Introduction to testing serverless applications | Hunting for errors
Marcia Villalba gives you an introduction into how we might test serverless applications and why flipping the testing triangle on its head might be a really good idea.

When you’re wondering if the developers at AWS ever sleep… 😴

Amazon Textract – Now Generally Available
Textract is a managed machine learning service that automatically extracts text and structured data from documents. It’s darn pretty cool, and I can image this saving quite a bit of data entry for all those companies and government agencies that still love paper forms. 🤖

Amazon Elasticsearch Service now supports SQL querying
So, this is actually quite amazing. Now you can query your Elasticsearch domain using standard SQL syntax. Plus, there’s a JDBC driver for it. Elasticsearch is crazy powerful, but the query language for it is a bit bulky. This should make it a lot more accessible.

Amazon Aurora Serverless MySQL 5.6 Now Supports Data API
Already mentioned this one, but it’s totally worth repeating. Still probably not fast enough for synchronous operations, but there are plenty of use cases that this is a perfect fit for.

Amazon Aurora with PostgreSQL Compatibility Supports Database Activity Streams For Real-time Monitoring
Yeah, so the cool stuff just keeps on coming. If you have compliance requirements (or you’re a fan of DynamoDB streams), you can now stream your database activity to CloudWatch (then off to Lambda), Imperva’s SecureSphere Database Audit and Protection, McAfee’s Data Center Security Suite, or IBM’s Infosphere Guardium. 😎

AWS Step Functions Enables Access to Workflow Metadata
Step Functions are no longer black boxes powered by elves living in a tree. You now have access to metadata about workflow executions, making it easier to correlate logs for faster debugging and to measure workflow performance data.

Amplify Framework Adds Support for AWS Lambda Functions and Amazon DynamoDB
The Amplify CLI now includes support for adding an AWS Lambda function to your AWS AppSync API using the GraphQL Transformer and the new @function directive. Lots of cool things you can do with these updates. More info in this blog post.

Announcing Tag-Based Access Control for AWS CloudFormation
I’m really loving this tag-based access stuff that AWS is doing. With the addition to CloudFormation, you can now deny certain users deletion or update privileges to stacks with a specific tag, while allowing access to others. 🔒

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

June 5, 2019 – BuildOn | Securing Serverless Apps on AWS with Alex Casalboni

June 11-12, 2019 – Designing Serverless Architecture with AWS Lambda with Yan Cui

June 17, 2019 – Premiere episode of the Serverless Chats Podcast. 🙂

June 21, 2019 – ServerlessDays Milan.

June 25-26, 2019 – AWS re:Inforce. I’ll be doing a Dev Chat on Serverless Security.

July 11, 2019 – ServerlessDays London

July 11, 2019 – AWS Summit New York

August 27, 2019 – ServerlessDays Sydney

August 29, 2019 – ServerlessDays Melbourne

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Michael Hart (@hichaelmart). Michael is the Director of Research Engineering at Bustle and an AWS Serverless Hero. Not only is he the creator of the popular LambCI tool, but he has several other open-source projects including docker-lambda, which was utilized to create AWS SAM Local (which eventually became the AWS SAM CLI). Recently, he wrote an eyeopening post reviewing the AWS Lambda nodejs10.x runtime, plus his Massively Parallel Hyperparameter Optimization on AWS Lambda piece is still one of my favorites. I’m pretty sure that everyone in the AWS serverless community has benefited from Michael’s work in one way or another. And for that, we thank you! 🙌

Final Thoughts 🤔

There are so many amazing things happening in the serverless ecosystem right now. The recent acquisition of Puresec (for $47M!) should be a clear market indicator that will push for more investment and more innovation in the space. The cloud providers are releasing new products at a mind-blowing pace, with libraries, frameworks, and downstream providers rapidly innovating and maturing in near-realtime. I remember the early days of cloud adoption, but never have I seen something like serverless that has redefined modern applications so quickly. This is a great show to watch! 🍿

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none.

If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless. 👍

Cheers,
Jeremy