November 30, 2021
Last week, pre:Invent overwhelmed us, Cloudflare tried to take over the serverless world, and Vercel raised another huge funding round to keep the innovation coming. This week, we’re in the throes of AWS re:Invent, we’ve checked off some serverless #awswishlist items, and I share my passion for Amazon DynamoDB. Plus, we have a TON of AWS announcements to cover and plenty of great content from the community.
Monitor & Debug Serverless with 30-Second Auto-Instrumentation
Complete observability over your serverless environment with no code changes required. With one-click distributed tracing, Lumigo lets you effortlessly find & fix issues in serverless and microservices environments. Get serverless-specific smart alerts before they impact performance or cost. Sign up for a free account and get up & running in minutes.
Most of you probably know how much I love DynamoDB, and more specifically, how much I love single table design. Well, I’ve decided to create a DynamoDB Modeling course to share all the tips, tricks, and strategies I’ve used and developed over the years. The course will launch in early 2022, but you can sign up to get updates (and early access) at dynamodbmodeling.com.
And because I like to go over the top, I created a DynamoDB parody song and video to the tune of Help! by The Beatles – My Database Needs Help! I hope you enjoy it!
Okay, here are some of the most existing serverless announcements from the past week. And some of them are BIG! Also, use this post to get a regularly update list of all the top announcements from AWS re:Invent 2021.
Announcing Amazon Kinesis Data Streams On-Demand – This one is huge, not just for production environments that need to scale up and down, but also for DEV and sandbox environments. This gets us closer to being able to test everything in the cloud without incurring charges for provisioned Kinesis shards! Marcia Villalba gives some more context here!
Amazon S3 Event Notifications with Amazon EventBridge help you build advanced serverless applications faster – Also huge! Now we can subscribe to events from S3 using EventBridge, allowing us to use all the amazing routing/retrying/filtering capabilities to fan out and trigger other services including pushing this info to API destinations, SaaS providers, and other accounts. More on this from Jeff Barr.
AWS Lambda now supports event filtering for Amazon SQS, Amazon DynamoDB, and Amazon Kinesis as event sources – Bigger than huge! EventBridge stepped in as a workaround for this very problem, but added extra Lambda Functions and more complexity when most of the time it just wasn’t needed. I did confirm with the Lambda PMs that each filtered Lambda requires a separate connection, so if you are filtering on small bits of data off a Kinesis stream for example, you could theoretically hit the Read Throughput limits. Although, with On-Demand Kinesis, who knows? A few people have written some posts showing how to trigger Lambda Functions with event filtering, or how to filter DynamoDB streams before a Lambda Function, and even how to do event filtering for Lambda Functions using AWS CDK.
AWS Lambda now supports partial batch response for SQS as an event source – Love this one! Now instead of failing an entire batch, you can just send back the failed message IDs. It will then just re-queue the failed messages and presumably follow your redrive policies for Dead Letter Queues (DLQs). And the Serverless Framework already has support for it!
Amazon SQS Announces Server-Side Encryption with Amazon SQS-managed encryption keys (SSE-SQS) – Maybe seems small, but nice to have the option to not need to manage your own encryption keys for SQS queues.
Announcing Amazon Redshift Serverless (Preview) – This one has been needed for a long time! So many amazing things that can be done with Redshift, but again, it was way too expensive to be spinning up for serverless development environments. Game changer. Danilo Poccia shares some more insights.
They also introduced Amazon EMR Serverless in preview and Amazon MSK Serverless in public preview. I don’t really use either of these, but for those that do, I’m sure you appreciate these new serverless models.
And… there was an AWS Free Tier data transfer expansion now allowing 100 GB from regions and 1 TB from Amazon CloudFront per month. I guess we should probably thank Cloudflare for this one. 😬
There are more announcements to get to, but there were some really helpful posts from the community that need to get shared as well.
Brian Foody has a helpful piece that shows you how to (almost) unknowingly create an infinite loop using the AWS SDK and why idempotency is so import in your Lambda Functions.
Eoin Shanaghy shares three ways to read SSM Parameters.
The Dashbird put out an infographic on securing serverless architectures, and I’m a sucker for serverless security.
Christian Bonzelet had a great post a few weeks ago that compared performance between Step Functions and Lambda. He followed up with part 2 that revisits the topic with cost comparisons as well.
Harshit Sharma shows how serverless can be a savior, even when using it as a peripheral service to your traditional infrastructure.
Notifications That Don’t Suck! The Product Hunt Newsletter recently covered a story on a company trying to fix notifications. Many apps that we use and download end up spamming us with notifications ruining the user experience. Similar to Wuphf.com created by Ryan on the hit TV show The Office, we are getting hit on every channel. How do we fix it?
Richard Rhodes shows you how to use the Serverless Framework to create an event-driven architecture.
Robert Bulmer explains how to implement rate and max depth limiting with AWS AppSync.
Here’s an interesting tutorial on implementing dynamic pipeline orchestration using CodePipeline, CodeBuild and Step Functions.
Emma Moinat shows you how to build an OpenSearch Index from DynamoDB streams.
If you still need to emulate the cloud locally, here’s how you can setup LocalStack for your local environment.
You can also deploy a containerized Serverless Flask app to AWS Lambda, if you really wanted to.
Kirk Kirkconnell shares a simple trick to identify items in an Amazon DynamoDB table that haven’t been accessed for a time and move them to Amazon S3.
And last but not least, Adrian Hornsby adds another post in his series on injecting custom faults with AWS Fault Injection Simulator. My advice is just to do whatever Adrian tells you. 😉
According to The Serverless Edge, you can deliver sustainable technology through Serverless.
And according to Snyk, cloud security is shifting to ‘dev’ not ‘ops’. Yep, some of use have been saying this for a long time. Maybe nobody was listening? 🤷♂️
Matt Asay wrote some follow up commentary on that whole back and forth about the unfulfilled potential/promise of serverless.
AJ Stuyvenberg explains why you should be using DynamoDB Streams and Lambda in your serverless applications.
And Ben Kehoe had a post from a few weeks ago explaining why AWS CodeArtifact should be the place to store AWS code artifacts. Couldn’t agree more!
On Serverless Chats Episode #121: Educating Serverless Developers, Rebecca and I chat with Ivonne Roberts about her new role at Bill.com, how she transitioned into tech and became an AWS Serverless Hero, what drew her to serverless in the first place, why making content is so important for your career, and much more.
Marcia Villalba shows you how to do faster serverless deployments and test your apps in the cloud with SAM Accelerate. I love that AWS is doing this!
Here is a dump of some other AWS announcements that look really interesting:
Okay, sorry for the firehose again. Hopefully there are some things in here that will help you build better, more amazing serverless applications!
I need to get back to re:Invent, but if you’re here and want to say, “Hi!”, DM me on Twitter.
Hope to see you,
Stay up to date on using serverless to build modern applications in the cloud. Get insights from experts, product releases, industry happenings, tutorials and much more, every week!
We share a lot of links each week. Check out the Most Popular links from this week's issue as chosen by our email subscribers.
Check out all our amazing sponsors and find out how you can help spread the #serverless word by sponsoring an issue.
Jeremy is the GM of Serverless Cloud at Serverless, Inc. and an AWS Serverless Hero that has a soft spot for helping people solve problems using serverless. He frequently consults with companies and developers transitioning away from the traditional “server-full” approach. You can find him ranting about serverless on Twitter, in several forums and Slack groups, hosting the Serverless Chats podcast, and at (virtual) conferences around the world.
Off-by-none is committed to celebrating the diversity of the serverless community and recognizing the people who make it awesome. If you know of someone doing amazing things with serverless, please nominate them to be a Serverless Star ⭐️!