Off-by-none: Issue #39

May 28, 2019

One small step for AWS, one giant leap for Step Functions… 👨‍🚀

Welcome to Issue #39 of Off-by-none. Thank you so much for joining us! 🙌

Last week we introduced the new Serverless Chats Podcast and took on some serverless contrarians. This week, we talk about a nifty little update to AWS Step Functions, share some serverless stories, and narrow down the mountain of content from the community. 🏔

Lots of serverless goodness this week, so let’s get to it. ⚡️

When you really wish you could get a callback… ☎️

This past week, AWS Step Functions added support for Callback Patterns in Workflows. If you’re not familiar with Step Functions (or state machines in general), they are a really useful way to orchestrate tasks in distributed systems. With serverless, they allow you to easily “compose” multiple functions that automatically trigger and track each step, as well as handle retries. This allows the “steps” in your application to execute in order, every time.

There are some issues with Step Functions (like pricing), but one of the biggest problems was controlling execution when long-running, synchronous tasks were required. This meant that any time you needed to pause the state machine, you had to set a delay that polled a resource to see if it could move to the next step. This worked well for shorter tasks, but what if you needed to wait several minutes? Or hours? Or days? You would end up paying for hundreds or thousands of executions just to check a status. Not very efficient or cost effective.

With the new Callback Pattern support, you can trigger a task and pass along the generated taskToken. The Step Function execution will pause until another process calls the SendTaskSuccess command with the taskToken to restart the workflow from where you left off! I can envision use cases that trigger third-party APIs that listen for callbacks on a webhook endpoint, or long-running transcoding/processing tasks that trigger callbacks from completion events. Think of all the possibilities (and cost savings)! Very cool stuff. 👊

Serverless News & Product Announcements 📢

Introducing Logbird — log gateway for the cloud
The Dashbird introduced a service to aggregate and filter log streams for events and trigger activities from them. Think of it as a centralized log collector that you can subscribe to for post-processing in event-driven applications. Interesting idea.

Log Search: Finding the needle in the haystack with Thundra!
Thundra’s new Log Search feature allows software teams to search through invocation logs using specific patterns.

Introducing New Relic Monitoring for AWS Lambda
Monitor, visualize, troubleshoot, and alert on your AWS Lambda functions using the New Relic ONE platform. Similar instrumentation and implementation as other serverless observability offerings.

ServerlessDays Stockholm – Call for Papers Now Open
ServelessDays Stockholm is scheduled for October 24, 2019. CFP is now open. Learn more at stockholm.serverlessdays.io.

Serverless Stories 📖

Lessons Learned From Sending Millions of Serverless Webhooks
Rocky Warren shares 10 things his team at Dwolla learned from converting their webhook notification system to serverless. Key insights in here regarding dynamic provisioning and resource tagging.

Less is More: Securing Payment Application with Serverless Architecture
Great post that shows how the Macquarie Group removed 60 groups of EC2 instances and replaced them with just a few AWS services including AWS CloudFront, AWS WAF, AWS Lambda@Edge and Amazon S3.

How we used the Slack API and Cloud Functions to help us keep track of events in our platform
Shailen Naidoo from GetTruck explains how they built a simple serverless Slackbot using Google Cloud Functions and Whatsapp.

Openfit: Serverless, CI/CD, SRE, Monitoring, and more
Read how API Gateway, Lambda, and DynamoDB, proved to be a perfect architectural match for Openfit’s requirements.

Serverless Use Cases 🗺

API Gateway to front services not hosted in AWS
Richard Boyd from iRobot is a big fan of API Gateway Service Integrations. In this post, he shows you how to use API Gateway as a proxy to external APIs without using Lambda. If you can wrap your head around it, you can use this type of pattern to cut latency and cost.

Real-time Streaming for Data Science with Time Series and Serverless
Data Science with serverless tends to get a bad wrap because of the limitations of certain providers. This tutorial from Iguazio demonstrates the use of Kafka and Nuclio to support the availability of streaming data in real-time datasets.

Deploy serverless Symfony on AWS Lambda with Bref
Want to make your Symfony PHP app serverless? Thibaud Lemaire outlines the use case and how to do it in this step-by-step post.

Automate Analyzing your IAM Permissions using IAM Access Advisor
Here’s a great little use case by Hassan Murtaza that uses generate_service_last_accessed_details and get_service_last_accessed_details to detect old IAM roles.

When you’re thinking about going serverless… 🤔

A serverless glossary
Just getting into serverless? Check out this glossary of terms by Toby Fee.

Testing Applications Built on Serverless Architecture: Don’t Fear the Transition
Dan Widing from ProdPerfect outlines how serverless changes (or doesn’t change) your testing practices.

15 Frameworks for Creating Serverless Apps
Alex Ivanovs lists 15 frameworks and tools that can be useful when building serverless applications. There are a lot more than this, but this should get you started.

An Introduction to Serverless Functions with The Minions!
Explain it to me like I’m a five-year-old. 😁 But seriously, if you’re new to serverless and you like Minions, this article will give you a basic intro to how Functions-as-a-Service (FaaS) works.

Serverless Tutorials 🏗

Safe List updates with DynamoDB
Robert Zhu gives you some tips and tricks to update lists in DynamoDB and avoid race conditions.

How to log timed out Lambda invocations
Yan Cui shows you how to preemptively log timeouts in your Lambda functions if you want something more than the standard “timed out”message.

Getting Started with the AWS Websockets API
An introduction to the power of an AWS Lambda powered real-time AWS WebSocket API, complete with Node.js code examples.

Adding Websockets to your AWS Serverless application
In case you’d prefer Python examples for using Websockets, Nino van Hooff has you covered.

Building a Serverless Workflow using AWS Step Functions
Dhaval Nagar takes you through building a simple workflow using AWS Step Functions.

Advanced Serverless CICD – Part 1 :  AWS CodeBuild
Erez Rokah shows you how to set up a Serverless CI/CD system using AWS CodeBuild connected to a GitHub repo.

AWS Lambda Layers: Getting Started Guide
Lambda Layers are extremely powerful and useful tool for your serverless applications. In this post, Ran Ribenzaft demonstrates how to create a layer with the Serverless Framework and then include it in your next project.

Serverless Security 🔒

Council Post: Rethinking Security For Your Serverless Applications
I think Ameesh Divatia’s recommendations might be overkill, but there are some good thoughts in here about the potential for human error.

5 Serverless Security Platforms
I agree with Ishan Mathur that developers need to be aware of security requirements, and having a tool that can help account for human error and nefarious behavior, can definitely provide that peace of mind.

4 fundamental practices for serverless architecture security
Clive Longbottom points out a few recommendations for securing your serverless applications. Most of these are best practices for any type of application, but always good to be reminded every once in awhile.

Serverless Reads 🤓

Reviewing the AWS Lambda nodejs10.x runtime
Michael Hart from Bustle banged up against the new AWS Lambda Node.js 10.x runtime, and he has just a few concerns. 😳

Getting it right between EC2, Fargate and Lambda
Choosing the right cloud compute service means balancing operational burden with cost to achieve the optimal experience. Sarjeel Yusuf has some thoughts on how to make the right decision.

Azure Functions’ Durable Entities
Chris Gillum from Microsoft runs through the thinking and use cases behind the new Durable Entities. I think there are times when rehydrating state would be very useful, but I’d hate to see it become a crutch that allows developers to embrace old, less-scalable application patterns.

4 reasons to go multicloud according to tech experts. Do you agree?
NO. 🤨 TLDR; 1) Vendor lock-in. Nope. 2) Customized solutions. Really? 3) Minimizes risk. If by “cloud” you mean a “server in a garage with a T-1 line,” sure. 4) Cost savings. If you ignore the cost of setting up and maintaining the complexity of multi-cloud as well as the lost revenue by over-engineering a solution and not getting to market fast enough. Other than that, I have no strong opinions on the matter.

5 Cost Saving Tricks to Reduce AWS Cloud Hosting Bill
If “cost savings” is making you think multi-cloud, try following Mark Runyon’s advice first. You know what’s a big cost saving? Going serverless. 😜

4 factors to ruin your serverless migration
Sure, serverless isn’t all 🦄s and 🌈s, but I have to respectfully disagree with Lee Atchison on a number of these points. You should certainly understand the performance capabilities of FaaS and whether it can handle your use case, but best practices and tools have emerged for managing complexity and controlling scaling costs. Plus, FaaS is the one part of serverless that has the least lock-in. Your data, on the other hand…

Data availability and backups in AWS
As Kirill Kolyaskin so eloquently puts it, “shit happens.” There are some rather interesting scenarios in here that could result in data loss, even if it’s “backed up.”

What the technology hype cycle tells us about the future of crypto
Tim Wagner has some interesting thoughts of where crypto is in the technology hype cycle, using his experience with AWS Lambda as a comparison. When the father of Lambda speaks, we should listen. 😉

Server to Serverless: How to achieve more by thinking of servers less
I really liked this piece by Richard Reukema as he points out that thinking about the underlying server deployments are a “distraction from the business need.” He says, “We need to stay above the implementation perspective, and only think of what the application requires from a services perspective.”

Thoughts from Twitter… 🐦

Lambda Layers Tip: I have a layer structure like this. It’s a utility layer for libraries I use (aws-sdk, aws-xray-sdk, uuid). I also have a simple response-handler.js. In Node I can use “const rh = require(‘/opt/response-handler’);” to utilize. #Serverless #ServerlessForEveryone ~ Eric Johnson
This is a neat way to utilize multiple libraries in a single require.

Not pleased to learn that the Node 10.x runtime for AWS Lambda is no longer logging serialized errors. Example code with screens of error output in 8.10 vs 10.x: exports.handler = async (event) => throw Error('simple err') This is really bad for bug resolution. 😠 ~ Brian Leroux
Brian is definitely not happy about this Node 10.x runtime for Lambda issue. I mean, look at how angry that emoji is!

Pearls of wisdom from @matthewfellows , “If you can’t build good microservices you shouldn’t be doing serverless” at @AWSMelb meetup ~ Tom Dickman
While I tend to agree, there are a lot of serverless use cases that don’t require as much choreography or orchestration as a traditional microservice would.

When you search for “serverless” on Netflix… 🍿

Building Event-Driven Serverless Apps with AWS Event Fork Pipelines
Stop reading and go watch this now! James Hood from AWS walks you through building event-driven applications and how you can use reusable Event Fork Pipelines to create loosely coupled serverless services. The underlying pattern is where the real power is.

AWS for Developers: Data-Driven Serverless Applications with Kinesis
Marcia Villalba has a new course out. It’s on LinkedIn Learning, so if you already have a LinkedIn Premium membership, you can access the course for free.

DevOps Chat: Serverless Intelligence With Lumigo’s Aviad Mor
Aviad Mor from Lumigo talks about how their customers are embracing serverless in hybrid environments, and why serverless is more about a way of thinking, rather than a specific technology.

Creating A Website Using AWS AppSync
Luke from Serverless Guru shows you how to build a file upload website using AppSync and DynamoDB.

American Heart Assoc: 100 years of Content is Discoverable with Serverless & AI on AWS
Heitor Lessa talks with AWS customer, Chirag Chhita, from the American Heart Association. He explains how they use serverless and AI tools to create better experiences for their platform users.

Azure Friday Live – Building serverless Python apps with Azure Functions
It looks like Microsoft bought an old CNN set to record this demo, but the fact that Azure Functions support Python now is pretty cool. Asavari Tayal takes you through a demo using the Azure Function VSCode extension.

Serverless Tools 🛠

cloudkj/scar: Deploy static websites in seconds – with HTTPS, a global CDN, and custom domains
Kelvin Jiang built a useful CloudFormation stack that allows “one-click” deployments for your static sites on AWS.

ICYMI: five updates you’ve missed about Serverless Step Functions
If you’re using the Serverless Framework and Step Functions, check out Yan Cui’s recent updates to the serverless-step-functions plugin.

When you’re wondering what AWS is building… 👷‍♀️

Updated timeframe for the upcoming AWS Lambda and AWS Lambda@Edge execution
Based on customer feedback, AWS has extended the timeframe for validation as well as given customers some more control. You now have five additional weeks to test and update affected functions.

CloudWatch Logs adds support for percentiles in metric filters
This is pretty cool. You can now visualize and trigger alarms on p90, p95, p99, p99.9 or any other percentile with CloudWatch logs.

Amazon DynamoDB adaptive capacity is now instant
This is another great update to avoid throttling on hot partitions. There are still limitations, so make sure to choose good partition keys.

Amazon API Gateway Now Supports Tag-Based Access Control and Tags on Additional Resources
And another really cool feature. Now you can use tags to limit access to routes/methods with API Gateway. I’m thinking this will be a very handy pattern for creating ACLs for Lambda and service integrations.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

May 29, 2019 – Build On Serverless, hosted by Heitor Lessa (with me as a guest)

May 29, 2019 – Webinar: Running Serverless with Full Observability (Thundra)

June 4, 2019 – ServerlessDays Tel Aviv.

June 11-12, 2019 – Designing Serverless Architecture with AWS Lambda with Yan Cui

June 17, 2019 – Premiere episode of the Serverless Chats Podcast. 🙂

June 21, 2019 – ServerlessDays Milan.

June 25-26, 2019 – AWS re:Inforce. I’ll be doing a Dev Chat on Serverless Security.

July 11, 2019 – ServerlessDays London

July 11, 2019 – AWS Summit New York

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Matthieu Napoli (@matthieunapoli). Matthieu is a software consultant and full stack developer that has been instrumental in introducing serverless to the PHP community. He is the creator of Bref, a PHP framework for building serverless PHP applications on AWS Lambda, as well as the creator of several other open source projects for PHP developers. Plus, he’s an active speaker at conferences, spreading serverless to the masses (of PHP developers). You can learn more about his upcoming speaking engagements and read his blog at mnapoli.fr. Thanks for your contributions, Matthieu! 🙌

Final Thoughts 🤔

The amount of serverless content being produced every week is overwhelming. But I really do enjoy reading it, picking out the interesting stuff, and sharing it all with you. If you find this newsletter valuable, and you think that some of your friends and coworkers might as well, I would really appreciate if you shared it.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none.

Until next time,
Jeremy