Off-by-none: Issue #88

May 5, 2020

API Gateway deserves some love too… 😢

Welcome to Issue #88 of Off-by-none. Thanks for being here!

Last week, we had too many releases from AWS to list, some really great EventBridge posts, and a debate about where serverless is in the technology adoption lifecycle. This week, we explore the super powers of API Gateway, answer the question of whether microservice size matters, and get a new song about S3 stuck in our heads. Plus, we have lots of content from our amazing serverless community.

Serverless News & Announcements 📣

Monitor all your Lambda functions, API Gateways, SQS queues, DynamoDB tables, and more, straight from Datadog
Natively create real-time serverless metrics at one-second granularity for your errors, cold starts, memory used, estimated cost, and more. Datadog’s AWS Lambda integration enables you to monitor all of your serverless metrics, plus logs and traces in one, unified platform, so you can get full context when troubleshooting an issue. Try it for free today. Sponsored

Serverless Events
AWS Serverless Hero, Gunnar Grosch, has put together a curated list of upcoming serverless events. If you know about an upcoming conference, meetup, podcast, webinar, or other awesome serverless event, make sure to add it.

A brand new primitive for your Begin apps: event functions!
I really like the simplicity of the Architect Framework, and Begin has been adding some pretty cool features lately. This new “event” primitive should make building your event-driven systems even easier.

And… A bunch of the usual suspects achieved the new AWS Lambda Ready designation. Congrats to all these excellent companies: Lumigo, Thundra, Epsagon, Dashbird, and Stackery (and any others I may have missed).

When API Gateway doesn’t want to be left out…

There were several posts this week that pointed out the benefits of using API Gateway service integrations to bypass Lambda. And since we called out AppSync a few weeks ago, I figured API Gateway deserved some attention as well.

Using API Gateway to run Database Queries
Renato Byrro advocates for using a service integration to query data directly from API Gateway to DynamoDB. Benefits included.

AWS API Gateway integration with DynamoDB
Hareesh Veduraj has a similar post to Renato’s, but this one goes into more detail and shows you how to set up a service integration to retrieve data from DynamoDB.

Don’t use Lambda to move data! API Gateway can help
And Richard Fan goes a step further and shows you how to put data INTO DynamoDB, complete with permissions and input validation.

Can AWS API Gateway Act as a Load Balancer?
And rounding out our API Gateway section, is another post by Renato Byrro, this time comparing ALB to API Gateway. Strangler patterns apply.

Serverless Stories 📖

Your Worst-case Serverless Scenario Part III: The Invisible Process
Niels van Bree wraps up his excellent series on how one mistake turned 100k records into 56M records and brought down their production environment. This post talks about the effects of partition key design and index replication.

Using DynamoDB as a calculator
This is what happens when you indulge Corey Quinn’s curiosity, a bit of insanity, but also what turned out to be a brilliant experiment. Alex Chan wrote a DynamoDB calculator, a self admitted terrible idea, but certainly worth the effort.

Serverless Use Cases 🗺

Continuous Deployment with Semantic Release and GitHub Actions
Brett Andrews shows you how to automate deployments and changelogs using Semantic Release and GitHub Actions. I love the idea of this, though I’m still not sure I’m ready to fully trust my semantic versioning and NPM publishing to an automated process.

Backing up Auth0 to GitHub using AWS Lambda
Here’s a straightforward, highly adaptable use case by John Wheeler. Use AWS Lambda to backup third-party service configuration information.

Dog Bark Detector – Serverless Audio Processing
Can you think of a better serverless use case than detecting when your dogs bark? Well, maybe, but this post by Nathan Glover will give you some really good insights into processing audio with serverless.

Making the Case for Serverless Use Cases
This is an article I wrote as a follow up to my Serverless Chats conversation with Gareth McCumskey. Newcomers to the space often don’t realize just how many use cases there are for serverless. This post makes the case for nine of them.

A Serverless Cloud for Stateful Applications
Rodric Rabbah (one of the creators of Apache OpenWhisk) took a slightly different approach to serverless with his company Nimbella. In this post he shows how to create a stateful stock trading application.

Serverless Concepts 🏗

Leveraging Lambda Cache for Serverless Cost-Efficiency
For most experienced AWS serverless devs, using global variables in our Lambda functions is second nature. If you’re still not sure what global variables afford you, take a look at this post by Renato Byrro.

Aurora Serverless: Data API vs Connection pools
I thought this was an interesting approach by Murali Allada. I really love the Data API, but I can see using it for user-facing, synchronous operations being a problem. But why not just use it where it makes sense?

How to trim your cloud infrastructure costs
Being cost-conscious is never a bad thing, especially when it comes to the cloud. This becomes even more important due to the economic impact of the current COVID-19 situation. This article had a few interesting cost-cutting measures that can hopefully save you a few bucks so you can better weather this storm.

13 Most Common Google Cloud Reference Architectures
Priyanka Vergadia compiled a list of her #13DaysOfGCP mini series on Twitter. In each video, she outlines a use case and a reference architecture using GCP. These might not all be applicable to you, but you know me, I’m a sucker for serverless patterns.

Deploying Microservices Using Serverless Architecture
I’m not 100% in sync with Dorival Querino’s depiction of using a Lambda function as an orchestrator in a serverless microservice architecture. However, assuming there was a message bus or Step Function to bridge that gap, the overall intent is similar to my thinking on the subject.

Serverless Tutorials 👷‍♀️

Creating a scalable serverless import process for Amazon DynamoDB
Great post by James Beswick that shows you how to use S3 as a staging environment to import datasets of any size into DynamoDB. This is a very useful pattern for either preprocessing data or database migrations.

Creating a local testing environment for AWS Lambda with Express and Typescript.
Joan Gil shows you another way to test your AWS Lambda-backed API locally. This example uses a combination of swagger-routes-express and the AWS SDK to load your API Gateway routes. It’s an interesting, albeit, complicated approach, but it apparently works for them.

Developing a Serverless Backend API using Flask
If you’re looking for a way to quickly make your Flask app serverless, this tutorial from Sourabh Mokhasi should get you there. This is an easy onramp to serverless, but once you’re up and running, start thinking about strangling some of those routes to separate Lambda functions.

Measuring Performance with CloudWatch Custom Metrics and Insights
Michael Bahr has a great post that shows you four approaches to generating metric data. Each approach has trade offs, but the post outlines when and why to use CloudWatch Insights versus asynchronous log analysis.

AppSync: how to compare strings lexicographically in VTL
Yan Cui gives you a simple solution for lexicographical string comparisons in VTL. More importantly is the way he figured it out, by getting help from an amazing #serverless community.

Serverless Security 🔒

How Compliance Standards See Serverless in Production?
Ah compliance standards, the bane of almost every fast moving developers. The good news is, that with serverless, lots of these things are handled for you. Emrah Samdan has a great post that outlines some common standards and their considerations when using AWS Lambda.

Serverless Reads 🤓

How to Remain Agile with DynamoDB
Excellent post by Rob Cronin that points out three common hurdles that new DynamoDB developers often face. As he points out, there is a tremendous amount of agility available to you if you start with the proper techniques and use DynamoDB for what it was intended.

5 AWS Services You Should Avoid!
This article has been making the rounds, so without piling on additional criticism, I’ll just say this: if you use cloud services poorly, you’re likely going to be the victim of poor outcomes (and then write posts like this).

Do not follow Hype of Serverless Blindfolded
And while we’re dumping on serverless, I figured I’d include this piece from Ashish Singh. Besides the fact that his cost estimates don’t include the engineer(s) that needs to maintain his three t3.2xlarge EC2 instances, I’m thinking that 50M (25 second long) requests/month spread across three servers means each one has to handle ~160 request per second. Probably fine if the traffic is evenly distributed, but how often does that happen?

ServerlessDays virtual wrap up
I had a blast hosting the first ever ServerlessDays Virtual conference, and thanks to Nick Van Hoof, we have a great write up for you to enjoy.

Serverless Needs Standards to Be the Future of Application Infrastructure
Thoughtful piece by Gadi Naor that makes the point that serverless adoption is likely being hurt by the lack of standardization across serverless providers. While I tend to agree with his analogy to the early days of web standards, there was also a tremendous amount of innovation that happened during that time. Too quickly standardizing on something like Kubernetes, could prove to be terribly inefficient in the long run.

Serverless Music, Podcasts, and more…

Durably: The Ballad of S3
Every day it becomes more and more apparent that Forrest Brazeal has few limits. He says he can’t dance, but that remains to be seen. 🕺

Serverless Chats – Episode #47: Programming AWS Lambda with Mike Roberts
In this episode, I chat with Mike Roberts about his new book, why Java shops should be just fine moving to AWS Lambda, how we need to approach testing serverless applications, and much more. Watch on YouTube.

How Big Should Microservices Be?
Not really about serverless, but whenever Sam Newman talks about microservices, I like to listen. I think this is probably the most common question, and while Sam’s answer might not be exactly what you were hoping for, it’s probably exactly what you needed to hear.

Observability in serverless applications – the Serverless Coffee Break
Ran Ribenzaft from Epsagon joins James Beswick to discuss how to bring observability to serverless applications, and shares some best practices in running serverless apps at scale.

Talking Serverless with Ryan Jones
Ryan Jones from Serverless Guru has started his podcast up again. Lots of great guests on here.

New from AWS 🆕

Amazon EventBridge schema registry is now generally available
The schema registry is such an awesome tool for all of us lazy developers that don’t want to document anything. And if you run this in your dev or staging environment, it’s basically free. Read more here.

Amazon Translate now adds support for Mexican Spanish
I know my Spanish isn’t very good, so let’s hope AWS can do a better job. And just in time for Cinco de Mayo!

Amazon S3 Batch Operations adds support for S3 Object Lock
You can now perform S3 Object Lock operations using Amazon S3 Batch Operations to apply or change legal holds or retention periods across many objects with a single API request or a few clicks in the S3 Management Console.

AWS WAF now supports migration wizard for converting WAF rules from AWS WAF Classic
I love easy upgrade paths, so it’s good to know that AWS WAF now provides a simple way to migrate your WAF rules from AWS WAF Classic.

Amazon Relational Database Service (RDS) Data API Client Library for Java now generally available
I’m a big fan of the Data API, so if you’ve been missing out because someone forced you to write your code in Java, at least now you have some good news.

NoSQL Workbench for DynamoDB adds support for Linux
Real devs use Linux! Not me, but this is something I (and apparently the NoSQL Workbench team) hear all the time.

Amazon CodeGuru Reviewer launches new, more cost-effective pricing model
I still haven’t played around with, but now revisions to a pull request are only charged for changed or newly added lines of code.

Serverless Tools 🛠

Aurora Serverless Data API with TypeORM
Lots of people have asked if you could use an ORM with the Data API, and now you can. Goje Dan shows you how to use the typeorm-aurora-data-api-driver package to integrate with TypeORM. And if you dig deep enough, you’ll find that it’s using my data-api-client package under the hood. 😉

Thoughts from Twitter 🐦

It’s that time again again. Sitting virtually with the Lambda PM + DA team talking about 2021. What do YOU want to @awscloud to build? #serverless #awsLambda #serverlessforeveryone ~ Chris Munns
If you have serverless requests, Chris is the person to send them to! Check out this post for some really great ideas.

Question for all you @dynamodb hackers out there. Is there ever a reason to store an attribute with a ‘NULL’ value, or should you just remove the attribute entirely? Please explain your answer. ~ Jeremy Daly
I always remove NULL attributes in DynamoDB, but apparently there are lots of people who see the value in storing them. Interesting discussion around this poll I posted.

Running a serverless insurance startup, selling home, auto, renters, and umbrella in five states (http://ourbranch.com ) on AWS. April AWS bill was just under $740. DynamoDB – $202 CodeBuild – $116 Cloudwatch – $100 S3 – $66 AWS Directory Service – $36 ~ Joe Emison
How much does it cost to run a fully serverless startup on AWS? Joe Emison gives you some of his numbers, with plenty of commentary from others.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

April 2-May 31, 2020 – Sessions with SAM: Live Twitch sessions by Eric Johnson (every Thursday at 10AM PDT)

May 7, 2020 – Building a social network in under 4 weeks with Serverless and GraphQL

May 8, 2020 – ServerlessDays Amsterdam 2018 (online edition)

May 14, 2020 – Virtual Meetup: Serverless Fail Day! 🗣

May 16, 2020 – Patterns and Practices for Building Resilient AWS Serverless Applications

May 21 & 28, 2020 – Serverless-First Function by AWS

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Belinda Vennam (@BeeMarieV). Belinda is an Advisory Software Engineer and Developer Advocate for IBM Cloud Functions and Apache OpenWhisk. Belinda led her team to deliver the functionality for the Templates feature in Cloud Functions, and has worked on and delivered a number of Open Source Apache projects, including a package for deploying serverless functions from a manifest file. Belinda has taken serverless cross-country, speaking at ServerlessDays Seattle and ServerlessDays NYC, among many other serverless events. Thanks, Belinda, for helping to make serverless better! 🙌

Final Thoughts 🤔

It looks like it was another busy week for all the stay-at-home serverlessers out there. I hope creating all this amazing serverless content is keeping you relatively sane. #StayAtHome is wearing really thin here at the Daly household, but we’re staying healthy. I hope you all are too.

Just an FYI, I’m planning on launching the next version of DynamoDB Toolbox this week, so stay tuned for that. Check it out if you get a chance.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none.

Until next time,
Jeremy