Off-by-none: Issue #63

November 12, 2019

The Fat Lambda Debate Rages On... ⚖️

Welcome to Issue #63 of Off-by-none. Thanks for spending some time with us! 😀

Last week, we heard about some exciting serverless investment/acquisition news. This week, we dive back into the #FatLambda debate, meet some new heroes, and of course, we’ve got a ton of great content from the serverless community.

🥳 Don’t forget! If you’re planning on going to AWS re:Invent this year, be sure to register for the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. We’re adding names to the waiting list, but we’d still love for you to join us and our awesome guests! We have some amazing sponsors (Stackery, Thundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless) that are making this event possible. I hope you’ll be able to join us!

Okay, let’s get right to it. Lots of great stuff to get to! 🏆

When you’re weighing the benefits of a single-purpose function…

Last week, I posted a tweet that outlined a sort of conundrum I was having with EventBridge. My latest project has multiple services subscribed to several events that use EventBridge as the messaging bus. This works really well, but my quandary was more about what the event processing architecture should look like. Should each service have separate Lambda functions to process each type of incoming event? Or should each service have just ONE Lambda function that processes ALL incoming events?

I added a poll to the tweet, and as you can see, team “One Lambda PER Event” beat out team “Fat Lambda” by a 3-to-1 margin.

So clearly we have a winner, right? Well, if you read through the thread, you’ll see a number of wise serverless sages make excellent points as to why Fat Lambdas might be preferable. As I mention in my tweet, single-purpose Lambdas let you isolate business logic, provide fine-grained IAM permissions, specify concurrency per operation, and more. All of this is at the expense of creating lots of configuration and boilerplate code to handle (sometimes) fairly trivial operations.

Tim Wagner pointed out that “at some point the CloudFormation outweighs the code, and skinny lambdas.” Tim Bray brought up the same issue as it applies to Step Functions, and wrote “if there are a bunch of different tasks to be done, do you make a new Lambda for each, or do you have just one function that examines the data and figures out what to do?” This is a valid question, especially given the level of complexity your organization is comfortable with. He also said that he’s “seen people go both ways and [hasn’t] been able to say that either is wrong.”

There were also others, like Gojko Adzic, who commented that he does neither. Instead, he “tend[s] to group events by security needs.” Heitor Lessa agreed, and gave a “+1 to that”, noting the importance of “Domain + Security context.” There were several others that simply touted the management simplicity of the #fatlambda, but plenty more, like Chris Munns, who argued the opposite and said he “think[s] longer term the flexibility will be non-blocking towards advancing certain capabilities, where as the fat func might eventually become overly complex and lead to you having to refactor too many bits at once.”

So who’s right? Team #fatfunction or Team #skinnyfunction? 🤷‍♂️ I hate to use the “it depends” answer, but there are certainly many factors that go into this decision when building serverless applications. The clear lesson here is that the community is still split on this. So, for now, whichever approach works best for you, is probably the right one.

Serverless News & Product Announcements 📣

Meet the newest AWS Heroes, including the first Data Heroes!
Yay! AWS has announced some new members to their Heroes program, and there are some great additions. Some of my favorites include Alex DeBrie, Farrah Campbell, Brian Leroux, and Gillian Armstrong! Congrats to all of you. 🎉

Tencent Cloud and Serverless Join Forces to Bring the Serverless Movement to China
Wow, this is pretty big. Tencent is the largest internet company in Asia and powers products like QQ, WeChat, Riot Games and League of Legends. Serverless, Inc. has made them a Premiere Partner, and you can already deploy an Express.js component to their Serverless Cloud Functions.

Fastly Announces Beta Version of the Compute@Edge Solution
Fastly is bringing serverless compute to the edge. According to their site, this is 100x faster than anything on the market. 🏎

Azure serves up pre-warmed Functions to offset cold starts
The Microsoft Ignite conference announced (or summarized) a few new features in their serverless arsenal, including their Premium plan supporting up to 60 minute execution times and pre-warmed instances. There’s also the ability to integrate secrets without needing to add any code.

Serverless Stories 📖

Architecting a Low-Cost Web Content Publishing System
Interesting story of how the University of St. Thomas used serverless technologies combined with legacy constraints to find a hybrid approach that solved their website speed, reliability, and management problems.

Serverless on Amazon: Is 2019 The Last Year I’ll Have to Deal With VMs?
Colin Dellow recounts three different types of projects he built using serverless and grades their overall performance. His approach to building an API and a website should make it obvious that more standard practices need to be followed (or maybe exist in the first place).

Top serverless plugins we are using
Hoang Le lists the Serverless Framework plugins that his team uses when developing serverless applications. It’s a long list, which probably shouldn’t be necessary, but there’s a nice serverless-mysql reference in there. 😉

Serverless Use Cases 🗺

An AWS Lambda ML Model Deployment
More interesting insights into deploying machines learning models on Lambda from Brian Schmidt.

Convert Radio Waves to Alerts using SDR, AWS Lambda and Amazon Transcribe
This is actually quite a fascinating use case (well, to me anyways 🤓). Using Software-defined Radio to capture audio from HAM radio frequencies, convert them to text using Amazon Transcribe, and then send alerts if certain things are said. That’s very cool.

Unlimited Email Sign-Ups for Free
Riccardo Giorato shows you a simple serverless use case, collecting email addresses from a web form with a serverless backend.

Processing High Volume Big Data Concurrently with No Duplicates using AWS SQS
Interesting post on how to use SQS polling and Lambda to reduce duplicate events. The problem I see here is that they’re using a Maximum Receive Count of “1” for an entire batch. This means that several messages in a batch could get processed successfully and still fail the Lambda, moving the entire batch to the DLQ. If you tried to replay those items, you’d likely reprocess some that already succeeded the first time.

Serverless Concepts 🏗

Containers vs/and Serverless
Emrah Samdan gives you the background on the difference between containers and serverless, what they’re good for, and when you might want to choose one over the other.

“Effortless” Serverless
Anjul Garg has a nice post that outlines some “best practices” for running serverless in production. Lots of good tips in here for people with all levels of experience running Lambda.

DynamoDB Importer
Awesome post by Rehan van der Merwe that shows you just how incredibly scalable DynamoDB is. He demonstrates the high throughput rate that DynamoDB can handle by writing 1 million records in 60 seconds with a single Lambda (~17k writes per second).

AWS S3 – 7 Useful CLI Commands
Quick little tutorial on using some of the standard S3 commands from the AWS-CLI. Never hurts to get a refresher.

The Resilient Architecture Collection
Adrian Hornsby has an excellent series on resilient architectures. If you haven’t looked at these yet, do yourself a favor, and find some time.

Serverless Tutorials 🏗

PHP and the AWS Lambda Custom Runtime (Part -1)
If you have any interest in compiling your own custom runtime for Lambda, this post by Mike McGrath might get you pointed in the right direction.

A Quickstart Guide to AWS Cognito, Lambda and SES
Ran Ribenzaft has a great post on getting started with AWS Cognito and integrating it into your serverless application. Writing your own authentication workflows is a lot of work (trust me, I know), so anytime you can use an out-of-the-box solution, it’s worth taking a look.

Writing an Alexa Skill with CRUD functionality using Dynamo DB, Serverless and AWS Lambdas
I honestly don’t think we utilize voice commands enough. Hamza Busuri has a full tutorial that shows you how to build a serverless Alexa skill.

Reduce bloat of your Lambdas
I’m always interested in optimizing my Lambda functions. Sérgio Ramos shows us an example of using the serverless-plugin-ncc to dramatically reduce package sizes and compilation time.

Serverless Security 🔒

Applying best practices for securing sensitive data in Amazon DynamoDB
You might want to grab your pocket protector for this one, because it’s for all us security nerds out there. 🤓 Really in-depth post with lots of really good pointers for securing your data in DynamoDB.

This Is How We Use Snyk to Protect Our Open-Source Projects from *Evil* Dependencies
Not specific to serverless, but third-party dependencies in our serverless apps are potential security threats as well. In this post, the team at Appwrite explains how they integrate Snyk into Github to scan for vulnerabilities on every pull request.

Serverless Reads 🤓

Why Serverless May Become a Dominant Programming Paradigm
I only included this because they say “Serverless is the new black”, which I thought was funny. There are some other good points in here too, but I think most of y’all already know them.

Going Serverless with AWS: Part 2
Eddie Jauregui has a simple explanation of a serverfull versus serverless workflow in an application that handles processing and resizing images. If you’re new to serverless, you might need to read the “Stateless and Event-Based Flow” section very carefully, but it’s a great explanation.

Most technology containers live less than five minutes, and lifespans are getting even shorter
According to this study, 63% of containers run for 10 minutes or less. Are these jobs that could be handled by FaaS?

Embrace the Creative Disruption of Cloud Native / Serverless
There’s a great tone to this post by John Gilbert. I liked how things used to work as well, but the cloud native / serverless approach, is just… better. Is it a paradigm shift? Yes. Is it “easier” to use? Not at first. But eventually it becomes just as easy as “the old days”, with all the benefits that come with this new approach.

For those that prefer the A/V club… 🎥

The Serverless Chats Podcast – Episode #22: Serverless Containers on Google Cloud Run with Bret McGowen
In this episode, I chat with Bret McGowen about how Google Cloud Run can make containers serverless, why Knative is important for building developer platforms, and what the future of serverless looks like at Google Cloud.

Building Resilient Serverless Systems with “Non-Serverless” Components
If you missed Serverlessconf NYC 2019, there are a lot of great talks to see. If you’re interested in watching mine, here it is. 😉

API Gateway Cognito User Pool Authorizer – Serverless Security
In her latest video, Marcia Villalba shows you how to set up a Cognito user pool authorizer for your API Gateway using AWS SAM.

Rethinking How You Do Development: Serverless at the Forefront
Danilo Poccia chats with serverless experts Austen Collins and Matt Weagle to discuss what it means to adopt a serverless mindset, how they got started, and some advice they have for anyone looking to get started with serverless.

Why Serverless is Cloud 2.0 – Serverless Computing London Keynote
Paul Johnston gave the keynote at Serverless Computing London and posted all 172 slides. There are lots of good points in here, and definitely worth taking a look.

When you’re curious what AWS is building… 👷‍♀️

NoSQL Workbench for Amazon DynamoDB adds support for DynamoDB local
You can now connect to your DynamoDB local instance using NoSQL Workbench.

Amazon QuickSight goes Mobile, launches Cross Source Join and More
Access to Amazon QuickSight from my mobile device? What is this, 2010? But seriously, this could prove to be very useful.

Amazon Comprehend Adds Six New Languages
Wow, this is big news for anyone doing NLP. Amazon Comprehend added six new languages including Chinese (Traditional), Chinese (Simplified), Korean, Hindi, Japanese, and Arabic.

15 Years of AWS Blogging!
Major props and thanks to Jeff Barr for all the work he’s done over the years sharing AWS knowledge. Crazy to think it’s been 15 years since he started blogging for AWS.

Introducing the AWS Step Functions Data Science SDK for Amazon SageMaker
This new open-source library allows you to easily create workflows that pre-processes data and then trains and publishes machine learning models using Amazon SageMaker and AWS Step Functions.

AWS CodeBuild Adds Support for AWS Secrets Manager
This is awesome. Much easier to pull secrets directly from Secrets Manager than to add them as environment variables referencing the Parameter Store.

Amazon CloudWatch launches cross-account cross-region dashboards
This might be one of the greatest features AWS has launched in a while. Accessing CloudWatch data across multiple regions and accounts will make monitoring and troubleshooting infinitely more simple. More details here.

Introducing notifications for AWS CodeCommit, AWS CodeBuild, AWS CodeDeploy, and AWS CodePipeline
I love the idea of notifications from these services, I just wish they were sent to EventBridge instead of SNS.

Thoughts from Twitter 🐦

That feeling when you’ve finally added proper DynamoDB indexes, refactored all Scans to Queries, cleaned up some old and unused stuff and your monthly AWS bill went down from ~$320 to ~$20. ~ Yury Smykalov
DynamoDB is an amazing tool, but as I (and hundreds of others have said), it’s not easy to learn. I love hearing stories like this that show the potential once people figure out some of the best/better practices.

CloudFormation: The Resource Import feature enables customers to import existing AWS resources into new or existing CloudFormation Stacks. ~ Jared Short
Jared’s keen changelog monitoring skills picked up on this gem. It’ll be interesting to see what sort of magical powers this will actually give us.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

November 14, 2019 – 5 steps to improving your software engineering team’s usage of serverless (Webinar)

November 19, 2019 – Thundra | Serverless at AWS re:Invent (Webinar) (I’ll be on this one)

November 22, 2019 – Is Serverless SecureLess? (Webinar)

December 2-6, 2019 – AWS re:Invent 2019

December 14, 2019 – ServerlessDays Fukuoka Japan

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Gillian Armstrong (@virtualgill). Gillian is a Solutions Architect at Liberty IT and a recently named AWS Machine Learning Hero. Gillian has been helping to bring machine learning and serverless into the enterprise by spearheading a number of internal projects, and then sharing her experiences and expertise with the community at large. Even though she is a “machine learning” hero, you’ll find her dedication to the advancement of serverless to be just as compelling. Thanks for all you do, Gillian! 🙌

Final Thoughts 🤔

There are so many new serverless things each week, that I find myself whittling through hundreds of posts, sometimes cutting out some really interesting takes. For those of you that are writing articles and sharing your experiences, first of all, thank you! The quality and quantity of serverless content out there is inspiring. And, secondly, if I don’t include your article, please feel free to reach out to me so that I can follow your work. One of the main reasons why I started this newsletter was to help others share their serverless ideas. If I can help you share yours, just let me know.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Until next week,
Jeremy