Off-by-none: Issue #95

June 23, 2020

Only have a few minutes? Check out this week's MOST POPULAR links as chosen by our email subscribers.

EFS + Lambda = Serverless for Everything? 🧮

Welcome to Issue #95 of Off-by-none. Thanks for being here!

Last week, we learned why now is the time to go serverless, saw how to build our own Chaos Monkey, and celebrated a year of Serverless Chats. This week, we see how EFS integration makes serverless even better, we get a lesson in why serverless is for everyone, and take a look at the internals of DynamoDB. Plus, we have a ton of awesome posts from the serverless community.

Serverless News & Announcements 📣

What Every CIO Needs to Know about Serverless
This TriggerMesh guide includes feature comparisons of major offerings and an introduction to foundational concepts like cloud native and microservices. Please check it out! We hope it can help inform your peers and managers on serverless benefits and available options. Sponsored

Salesforce introduces several new developer tools including serverless functions
I’ve said it before, and I’ll say it again, I really like how SaaS companies are empowering their users by giving them access to compute in their environments. So many interesting things that you can do, including maybe even just triggering a data export to your main application.

Fastly Achieves 100 Tbps of Edge Capacity Milestone
Pay attention to Fastly. The edge market is going to become more and more important for building cloud/serverless applications, and I think Fastly is positioning themselves nicely.

Announcing Epsagon’s Applied Observability for Azure and K8s
I’ve always been a big fan of Epsagon, and they continue to smartly expand their offerings. Getting into Azure and (cough) Kubernetes, is another good strategic move.

New Updates on Webiny 4.1.0
After the recent launch of the Serverless Headless CMS, the team over at Webiny is continuing to churn out new features and capabilities.

Datadog now supports Amazon EFS for AWS Lambda on Amazon Web Services
I think EFS integration with Lambda is going to be huge, so Datadog jumping on this right away is good to see.

AWS Adds EFS to Lambda… 😲

Last week (just after we published the newsletter 🤨) AWS released a Shared File System for Your Lambda Functions, that is, integration with Elastic File System (EFS). This is a very big deal and I think it opens up a lot of new use case for serverless (including machine learning). There were several well written pieces on the subject, so I’ve included some of my favorites below:

Migrating Image Memorability service to AWS Lambda using a shared Elastic File System volume
Excellent post by Luca Bianchi that shows you just how powerful (and cost effective) the new EFS integration with Lambda can be. This is also an excellent example of why rethinking your architecture to optimize for the final result is so important. The individual image processing times may have increased, but the parallelization significantly reduced the total job time.

Using Amazon EFS for AWS Lambda in your serverless applications
I loved this piece by James Beswick because it outlined some sample applications that you could build with EFS integration. It’s important to note that he shows “asynchronous” operations, not real-time synchronous ones that could suffer from EFS latency.

And here are a bunch more that are worth digging into:

• Unlocking new serverless use cases with EFS and Lambda by Yan Cui
• Get Started with EFS File System Access for AWS Lambda by Ran Ribenzaft
• Monolithic Ideas For AWS Lambda File Systems by Ken Collins
• How I used Lambda and EFS for massively parallel compute by Peter Sbarski
• How to use EFS with AWS Lambda? by Mohamed Latfalla

Serverless Stories 📖

Multi-Region Lambda with Route53
Dre May documents the experience of setting up a multi-region Lambda deployment using the AWS CDK. All the code is provided, but the article is a great read for understanding where you might run into rough edges, and how to deal with them.

Cloud-Native Development just clicked for me
Great perspective from Pablo Bermejo about why companies need to build cloud-native instead of trying to achieve some level of cloud/on-prem agnosticism.

My Experience with the Cloud Resume Challenge
Good on Forrest Brazeal for creating the Cloud Resume Challenge. I’ve see lots of people really pushing themselves to complete it and creating excellent posts of their journey, like this one by Will Boland.

Serverless Use Cases 🗺

Automated end-to-end observability for microservice environments
Epsagon enables full-depth observability of complex serverless and containerized architectures, delivering the automation needed to instantly identify, troubleshoot, and resolve issues before they affect production. Try free today. Sponsored

Implementing geohashing at scale in serverless web applications
Using location data within your mobile (or even web-based) applications is becoming more and more common. James Beswick outlines how using serverless tools like DynamoDB and IoT Core can make building and scaling these types of applications super easy.

Building a Github Webhook handler using AWS SAM
Straightforward use case by Sourabh Mokhasi. I’d think about using EventBridge instead of SNS (you’d get more targets that way), and if your target is another Lambda function, SQS might make more sense for better buffering and greater insight into the queue.

Serverless GraphQL server on AWS Lambda
This is a little bit use case, and a lot tutorial. John Dousaitis shows you how to set up an Apollo GraphQL “server” on AWS Lambda.

Serverless Concepts 🏗

Internals of DynamoDB
Excellent writeup by Udit Sharma that explains how the different parts of DynamoDB work in order to ensure high availability and scalability in your applications.

Using API Gateway to Decouple and Scale Serverless Architectures
Renato Byrro offers some thoughts into using API Gateway as an abstraction layer to other AWS services. I think there are a lot of interesting use cases for service integrations, but I’m not sure I fully agree with this implementation.

How to choose the right API Gateway auth method
Yan Cui gives you a handy guide to help you choose the appropriate API Gateway authorization method. There are a lot of options, including using direct integrations to third-party provides with JWT, but if you’ve got no idea where to start, this should point you in the right direction.

All you need to know about SQS and Kinesis is Here
Edward Huang gives you a breakdown of Kinesis and SQS and outlines some of the benefits and drawbacks of each.

How to test serverless workflows?
Joel Lutman explains why implementing regression tests in serverless applications are important, what advantages they have over unit and integration tests, and how to set your system up to automate them.

Serverless Tutorials 👷‍♀️

COURSE: Serverless for Frontend Devs in 4 weeks 🎓
Use code OFFBYNONE_JUNE for 10% off
Learn from AWS Serverless Heroes Slobodan Stojanovic & Alex Simovic. ~5hrs/week starting July 6th. You’ll learn to set up and secure Amplify & React using Cognito, connect GraphQL, manage permissions, use AppSync, test your code, and much more. Homeschool by Senzo is a virtual classroom giving you live chat with your classmates, weekly instructor Q&A + perpetual access to course materials. £/$400, €440. Sponsored

How to connect to MongoDB cluster from AWS Lambda?
Simran Kaur Kahlon shows you how to use host aliases when connecting to a MongoDB cluster from AWS Lambda. Seems like a straightforward solution.

Serverless database computing with Azure Cosmos DB and Azure Functions
Gwyneth Pena shows you how to set up an Azure Cosmos DB and then utilize it from Azure Functions using input and output bindings.

Jamstack and the power of serverless with FaunaDB
And incredibly in depth tutorial by Richard Haines that walks you through creating a Jamstack website powered by Gatsby, Netlify Functions, Apollo and FaunaDB. There’s the opportunity to swap in some of your favorites tools here, but this will give you a very good overview of how Jamstack sites are built.

Coordinate Complex Workflows with Step Functions
Serkan Özal gives you some tips on the when, why and how of Step Functions, and then shows you how to build a state machine that loads images from S3 and runs them through Rekognition.

A Basic CI/CD Pipeline for Serverless Apps
Emily Shea has a great tutorial that shows you how to implement a basic CI/CD pipeline using the AWS Serverless Application Model (SAM), GitHub, and CircleCI.

Serverless Reads 🤓

Serverless is for Everyone!
Another excellent read by Sheen Brisals that debunks a few common misconceptions about serverless, explains why adoption faces challenges from ingrained thinking, and offers a list of resources to help you get started on your serverless journey.

Why Do You Need a Backend?
I have been fascinated lately with the possibilities of the Jamstack, and seeing what’s available to frontend developers. Kay Plößer has another good post that asks the question, and then answers, why you do (or don’t) need a backend for your applications.

Serverless Adoption Survey — The numbers so far
Paul Swail started a research project into the gains and pains development teams encounter when adopting serverless in AWS. He’s got some interesting preliminary numbers to share with you.

How to write less code in your serverless application with declarative cloud functions
Never mind functions calling functions, what about functions creating functions? Shams Mosowi illustrates a pattern that uses generator functions to enforce consistent data flows. It’s an interesting approach, but as he points out, there are some drawbacks.

Podcasts and Videos… 🎧

Serverless Chats – Episode #54: Coordinating Cloud Engineers and Serverless Developers with Joe Duffy
In this episode, I chat with Joe Duffy about the biggest challenges for teams working in the cloud, how infrastructure-as-code (IaC) helps teams collaborate better, the distinction between serverless developers and cloud engineers, and a lot more. Watch on YouTube.

Observability and Monitoring in Serverless Applications
Marcia Villalba is joined by Danilo Poccia to discuss the new launches and functionality in Amazon CloudWatch and how you can use them observing serverless applications.

New from AWS 🆕

AWS Amplify Console now supports deploying and hosting web apps managed in monorepos
I was just looking for this a week or two ago, so they must have heard me. 😉 This is an awesome feature that allows you to specify a subdirectory in your repo and it will only detect and kick off builds if changes are made to that directory.

Introducing AWS Solutions Constructs
If you are building with the AWS CDK, things just got easier. There are over 20 pre-built, multi-service architecture patterns that allow you to quickly assemble well-architected applications. Read more here.

Amazon Polly launches a child US English NTTS Voice
Probably strange if you have a ten year old’s voice read you the latest technical blog post, but could be a great feature for applications that are geared towards children.

Introducing Multi-Region Application Architecture
More AWS Solutions to help you build better applications. This solution leverages Amazon Simple Storage Service (Amazon S3) Cross-Region replication and Amazon DynamoDB Global Tables to asynchronously replicate application data between the primary and secondary AWS Region.

Amazon SQS now provides results for the “List Queues” and “List Dead Letter Source Queues” requests in multiple pages
Probably not the most exciting announcement, but if you have a complex system with lots of queues and DLQs, having paginated access to them through an API can be very useful.

Introducing AWS Snowcone – A Small, Lightweight, Rugged, Secure Edge Computing, Edge Storage, and Data Transfer Device
This thing is pretty cool. I don’t have any immediate uses for it, but I can see some really good use cases, especially if you need to get massive amounts of data in and out of the cloud.

Amazon Aurora Global Database supports read replica write forwarding
If you’re still using RDBMS over DynamoDB and want to utilize global replications, Aurora Global Database is a good option. With this latest update, you can now simplify your application code and simply direct writes to the local region’s cluster. It will automatically forward the write requests back to the primary region.

Amazon SES can now send notifications when the delivery of an email is delayed
This is a cool feature, especially if your application has strict time requirements for email delivery. This would make it possible to use a secondary contact method if a valid email address has temporary issues.

Amazon API Gateway allows subprotocols on a WebSocket API connection
I remember this being a big problem if you didn’t necessary control the client, so now having support for subprotocols is a nice add-on.

New on Coursera: Digital course on building Amazon DynamoDB-friendly apps
If you’re looking to level up your DynamoDB game, this looks like a good course to check out. It says you can register for free, but there’s also something about paying for a certificate. Probably worth looking into.

Serverless Tools 🛠

cmda
cmda (think: commander) is a CLI tool by Michael Hart that can execute remote commands, including uploading local files/directories, to an AWS Lambda function. Along with EFS for Lambda, this makes it an easy tool to move files to/from an EFS file system that you might use with your other Lambda functions.

Serverless Jobs 👩‍💻 Sponsored

Serverless Engineer – stedi.com
At Stedi, we’re working in one of the biggest markets on the planet – EDI, the technological backbone of the physical product economy. We’re building a next-generation platform: a ubiquitous commercial trading network to automate the trillions of dollars in B2B transactions exchanged by nearly every company on Earth. If you’re interested in what we’re building and how we’re building it, we’d love to hear from you.

Have a job listing you’d like to share? Please contact me for more information.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

June 24, 2020 – Serverless Security Strategies: Under the Hood (AWS Fireside Chat)

June 26, 2020 – AWS Serverless Community Day

July 6, 2020 – Serverless for Frontend Developers (Course)

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Dr. Denis Bauer (@allPowerde). Denis is the Group Lead at Commonwealth Scientific and Industrial Research Organisation (CSIRO) and she’s been doing some pretty amazing things with serverless. As an internationally-recognized expert in machine learning, Denis leads the team at CSIRO in using AWS infrastructure to process genomic data to unlock secrets in human DNA, and using serverless architecture to build a web service that identifies the optimal genome engineering sites. She has developed an open-source, AI-based cloud service that accelerates disease research. Thank you, Denis, for your research and work, and contribution to both science and technology! 🙌

Final Thoughts 🤔

I’m really excited about this new EFS integration with Lambda. I’ve run a few experiments and had similar results to others in regards to performance. And even though we’re not getting mounted drive speeds, it is still fast enough to process most background tasks that require large file access patterns. It might not be perfect yet, but the ability to have shared access to a massive file system directly from your Lambda function is pretty darn amazing. If you build anything really cool with it, please let me know.

I hope you enjoyed this newsletter. We’re always looking for ideas and feedback to make it better and more inclusive, so please feel free to reach out to me via Twitter, LinkedIn, Facebook, or email.

Take Care,
Jeremy