Off-by-none: Issue #36

May 7, 2019

Developing serverless applications “locally”… 🏠

Welcome to Issue #36 of Off-by-none. It’s nice to see you all again! 👋

Last week we tried to wean ourselves off of our RDBMS addiction, and we explored some interesting serverless use cases. This week, we look at how we can create a better local development experience when building serverless applications. Plus we have a ton of amazing content from the serverless community.

Lots of great stuff to get to this week, so let’s get right to it. 👍

When you’re looking for a better local serverless development workflow… 👨🏻‍💻

The past couple of weeks have seen a number of updates that are trying to make developing and testing your serverless applications easier when working in a local environment. Stackery recently added cloudside support to their local developer tools, and Zeit announced now dev, a tool that performs “hot-reloading” of serverless API backends. These are all great advances, but I’m still a big fan of the Serverless Framework. And unfortunately, their local development experience is still missing a few pieces.

One of those pieces is the inability to seamlessly execute your code locally while accessing cloudside resources. This has been a major issue for me for a very long time (like since November of 2016). The workaround involved using pseudo variables to construct my own service URL or ARN. But this method was error prone and required hardcoding formats for the different service types. More recently, I would export the references in CloudFormation and then use the Serverless Framework cf variable format to load them dynamically. Usually something ugly like this:

QUEUE: ${cf:${self:service}-${self:provider.stage}.MessageQueue,"not deployed"}

So with all the recent hype around local development using cloudside resources, I decided to solve my problem (and maybe yours too) by creating the serverless-cloudside-plugin. With this plugin, you can now use standard CloudFormation intrinsic functions (like !Ref and !GetAtt) in your environment variables. These will automatically be transformed into proper references (like an SQS QueueUrl or an SNS Topic ARN) when executing or testing your functions locally. It even works with serverless-offline and test runner plugins.

If you want to learn more, read my new post Developing Serverless Applications Locally with the “serverless-cloudside-plugin” or check out the plugin on GitHub. I hope it makes your local development experience better, I know it works for me. 😀

Serverless News & Product Announcements 📢

“We recently shaved 1/3 off the p99 on the #awsLambda service’s overhead latency”
Chris Munns took to Twitter to share this pretty exciting announcement about updates that make Lambda functions startup faster. Another benefit of going serverless? Every function you have in production automatically gets these updates.

Serverless Framework Enterprise 0.9.0 – Error Insights
Serverless, Inc. keeps pushing some great new features to their Enterprise Framework. This latest update includes some really great tools that can surface new errors within your applications.

Build 2019: Microsoft, Red Hat develop open-source service for auto-scaling serverless containers on Kubernetes
Yes, there were a lot of announcements from Microsoft this past week. Here’s a good post that summarizes them all for you.

Introducing faast.js
According to the docs, faast.js is “Serverless batch computing made simple.” Actually might be an interesting idea; ephemeral batch jobs using a straightforward framework.

Introducing… CloudFormation extrinsic functions 😉
Yan Cui has a new Serverless Framework plugin that lets you use a number of custom functions anywhere in your serverless.yml as if they’re CloudFormation’s intrinsic functions. Some interesting use cases here.

Serverless Use Cases 🗺

Using CloudWatch and Lambda to implement ad-hoc scheduling
Yan Cui has another great post that shows you a few alternatives to creating serverless cron jobs.

Sharing AWS SNS Topics Across Stages
The engineering team at FloSports needed to decouple their SNS Topics from their serverless.ymls. This is an interesting approach.

Custom geo restriction of AWS CloudFront Distribution using Lambda@Edge
Lots of interesting use case with Lambda@Edge. Here, Rumble Fish used it to modify the behavior of CloudFront based on an IP address blacklist.

Running TensorFlow on AWS Lambda using Serverless
Mike Moritz outlines the process for building a serverless machine learning model.

Serverless “Not Hotdog”
Now you can have your very own Silicon Valley-esque “Hot dog, not hot dog” app. Peter Vanhee shows you how he built a serverless image classifier using fast.ai, AWS Sagemaker and Lambda.

Enterprise use cases for AWS Lambda
Data Lakes, microservices, and disaster recovery are just a few of the serverless use cases for enterprises. Mart Laul from Dashbird breaks these down for you.

When you’re still trying to figure out this whole “serverless” thing… 🤔

Serverless computing pros and cons: 5 benefits and 3 drawbacks
This is a good primer from Brandon Vigliarolo that outlines a few pros and cons of serverless. I wouldn’t quite agree with all of these, but it’s helpful nonetheless.

4 Serverless Myths to Understand Before Getting Started with AWS
To rebut some of Brandon’s claims, Tanusree McCabe from Capitol One has a great article that I whole-heartedly agree with.

Deconstructing Serverless Computing Part 2: the Good, the Bad, and the Time to Market
Lucian Toader does a deep dive on some of the practical use cases for serverless, and where it really shines for organizations.

Serverless Stories 📖

How Mutual of Enumclaw Transformed from AS/400 to Serverless with Confidence
Great story that outlines their approach to moving from an AS/400 and Z-Series Mainframe to running serverless using AWS Lambda.

Building a Text to Speech Converter with AWS Polly
Danielle Heberling spent some time building a serverless text to speech converter. This post shares what she learned during the process.

How I Built A Serverless RSVP Feature With API Gateway, Lambda, and DynamoDB
Emilia Rubio summarizes how she approached building a serverless RSVP feature for her brother’s wedding website and shares a few helpful resources she used.

Go global variable in AWS Lambda
João Reis had a few problems with global variables in his serverless application. His problem was an easy fix, but it might not be obvious to those new to serverless.

Limiting Down Time by “Swapping Stacks” in AWS Lambda Serverless
FloSports shows us another interesting method for minimizing down time with their serverless applications when rerouting traffic between deployments.

Serverless Tutorials 🏗

S3 static site hosting over https with an external domain
Mim Armand gives you “all you need to know and nothing more!”

Combining Terraform and the Serverless Framework Gracefully
Thales Minussi shows you how to build out your shared resources using Terraform, then combine them with your Serverless Framework project.

How to split your serverless.yml with shared resources when you hit the 200 resource limit on AWS CloudFormation
Here is Sharath Prabhal’s simple trick for getting around this limitation.

The Complete Guide to User Authentication with the Amplify Framework
Trying to building your own authentication system when you have Cognito is just nonsense. Nader Dabit gives you the details on everything you need to know about using this service with Amplify.

Automating Serverless framework deployments using Bitbucket Pipelines
Ayush Sharma shows you a simple way to deploy your serverless applications using the Serverless Framework and Bitbucket.

Node.js 6 is approaching End-of-Life – upgrade your AWS Lambda functions to the Node.js 10 LTS
Node.js 6 reached its EOL on April 30th! Liz Parody shows you how you can (and why you should) upgrade your functions to Node.js 10 using a custom runtime. I’m sticking with 8.10 for now.

Serverless Security 🔒

How To Handle Secret Management for Serverless Applications
Pete Hugh outlines the top 3 mistakes and best practices for Serverless Secret Management.

4 Reasons Why CISO’s Need to Have a Plan for Serverless Security in 2019
If you’re a Chief Information Security Officer and your company plans on going serverless, you might want to take a look at this post.

Broken Access Control in Serverless Deployments
According to Protego, over-privileged functions are everywhere, and if the attacker spots the right one, your whole cloud account is in danger.

Serverless Reads 🤓

A Guide to S3 Batch on AWS
S3 Batch is an amazing new service from AWS, and Alex Debrie gives you a complete guide to learn when, why, and how to use it. Check out his plugin for it as well.

Building Serverless Pipelines with Amazon CloudWatch Events
This is an awesome post by Forrest Brazeal that shows you how CloudWatch Events can be used as a serverless message broker similar to SNS. But with nothing to provision, and with similar costs and filtering capabilities.

To VPC or not to VPC? Pros and Cons in AWS Lambda
Yan Cui gives you some advice about using Lambda functions with VPCs. Bottomline, only use them when you have to.

Disaster Tolerance Patterns Using AWS Serverless Services
I love patterns! Ken Robbins has a really great post that outlines a number of patterns and practices to ensure that your service deployments are resilient.

Wide adoption of serverless will be led by tomorrow’s giants, not today’s
Erez Berkner says that soon serverless will be seen as a primary way to develop new applications. That means the next generations of startups will be “serverless first”, ultimately leading to widespread adoption. Makes sense to me.

Using IoT with Serverless to Tackle Global Issues
I really liked this post by Sarjeel Yusuf. IoT can be a powerful tool for social projects. But there are limitations (cost is one) when it comes to being able to collect and process all that sensor data. Sarjeel argues that serverless is a great solution to this problem.

The new AWS X-Ray Analytics feature: How your serverless apps can benefit from it
Nuweba breaks down the new analytics feature of AWS X-Ray and shows you some ways you can use it to get better insights into your serverless apps.

How to Get Real-Time Visibility Into Serverless Apps
Adam Johnson from IOpipe shares some of the challenges he has faced in this industry, along with best practices and technical tips when it comes to monitoring your serverless applications.

Thoughts from Twitter… 🐦

Comments like this are in line with my thinking that k8s is the last hill for traditional IT/Ops to die on. After this more fully managed services and #serverless will dominate. Rough edges today, yes, but so did “cloud” 7 years ago. ~ Chris Munns
Interesting take on a post by Kelsey Hightower. The pace of innovation is absolutely staggering. Serverless four years ago is almost unrecognizable today. And in a very good way.

If you think that building #Serverless Architectures on #AWS is “innovation” and a risky choice maybe you should take a look at this screen by Mai-Lan Tomsen: “Serverless Customers” ~ Sergio Lecuona
I still think building with serverless is “innovative”, but a risky choice it is not. You might recognize just a few of the companies already using serverless in the attached photo.

When you prefer a multimedia experience… 🎞

Deployment strategies for serverless applications: Canary and Linear
In this video, Marcia Villalba talks about the major deployment strategies for serverless applications and how AWS helps you implement them.

Episode 58: The Cloud Is Great, But It Isn’t Perfect | Screaming in the Cloud
Corey Quinn talks robots and serverless with Richard Boyd from iRobot in this episode of his podcast.

When you’re wondering what AWS has been working on… 👩‍🚀

Amazon Aurora Serverless Supports Capacity of 1 Unit and a New Scaling Option
Yup, I totally missed this last week. 🤦🏻‍♂️ Aurora Serverless now supports scaling down to just 1 ACU, which makes it more affordable for low traffic workloads. Check out my updated post on Aurora Serverless to learn all about it and to see how this update changes the math.

Amazon S3 Introduces S3 Batch Operations for Object Management
So S3 Batch is amazing. The use cases for processing existing S3 items is probably infinite. You can check out Jeff Barr’s post for more information on how to use it.

AWS Amplify Console adds support for Incoming Webhooks
This means you can trigger deployments without needing to commit code to a repository, which is perfect for integrating with a headless CMS. Check out this post to learn more about how it works.

AWS Amplify launches an online community for fullstack serverless app developers
This new community site is fully open source, allowing developers to contribute blog posts, videos, sample projects, and tutorials through GitHub pull requests.

Amazon Cognito launches enhanced user password reset API for administrators
The new AdminSetUserPassword API for the Cognito User Pool service lets administrators set temporary or permanent passwords for their end users.

SummitRoute/aws_breaking_changes
Speaking of changes, this new GitHub repo maintains a list of changes announced for AWS that may break existing code. S3 users be warned.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

May 8, 2019 – Stackery Livestream – AWS Serverless Application Patterns with Jeremy Daly (this should be fun)

May 14, 2019 – PureSec Webinar – PCI Compliance & Serverless: Everything You Need To Know

May 15, 2019 – Serverless Monitoring & Troubleshooting – Expert Roundtable with Yan Cui, Jeremy Daly and Erez Berkner 😊

May 21, 2019 – Epsagon Webinar: Modernizing Applications with Serverless on AWS

May 22, 2019 – Build On Serverless, hosted by Heitor Lessa (with me as a guest)

May 22, 2019 – Building Event-Driven Serverless Apps with AWS Event Fork Pipelines. Hosted by James Hood.

May 25, 2019 – Webinar: Comic Relief’s Journey to Serverless for Red Nose Day (IOpipe)

June 4, 2019 – ServerlessDays Tel Aviv.

June 21, 2019 – ServerlessDays Milan. The agenda was just released!

June 25-26, 2019 – AWS re:Inforce.

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is James Thomas (@thomasj). James is a developer advocate lead for serverless at IBM Cloud. He’s heavily involved in the serverless community as an organizer for ServerlessDays London (@serverlessldn), a member of the global ServerlessDays committee, and a contributor to the open-source Apache OpenWhisk project (@openwhisk). He’s also a regular conference speaker, active on Twitter, and always updating his blog with interesting serverless posts. Thanks for what you do, James! The community appreciates it. 🙌

Final Thoughts 🤔

It was another great week for serverless innovations. Between S3 Batch (which is amazing), all of the announcements from Microsoft, and, of course, some great new plugins 😉, we are moving closer and closer to mature tooling, powerful frameworks, and battle-tested backend services. The rough edges of serverless continue to get smoothed out, and it’s great to see so many people dedicated to making serverless the “primary way to develop new applications.”

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions. They help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) how you’d like to contribute to Off-by-none.

If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless. 👍

Until next time,
Jeremy