Off-by-none: Issue #65

November 26, 2019

Serverless is a journey, not an event destination… 🪐

Welcome to Issue #65 of Off-by-none. It’s great to have you here! 😎

Last week, we looked at the first major batch of announcements and feature launches leading up to re:Invent. This week, there were about a thousand more (give or take 😉)! We’ll try and cull that list down for you, plus we still have plenty of content from the serverless community.

Before we jump in, I just want to thank the sponsors of the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. Stackery, Thundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless have made this entire event possible.

Okay, now prepare for serverless announcement overload! 🤯

Guides to at Serverless re:Invent 🗺

re:Invent is going to be a nonstop onslaught of information, technical sessions, product announcements, parties and more. Here are some guides to help you plan your time. (BTW, if you’re not going to re:Invent, take note of some of these sessions and watch the videos once they’re posted.)

AWS re:Invent Serverless Survival Guide
Trisha Paine provides a list of can’t miss re:Invent sessions for developers, cloud architects, and security folks.

re:Invent Guide: Advanced Serverless
I’m including Eric Hammond’s excellent guide to Advanced Serverless sessions again. Great stuff in here.

Serverless at re:Invent
I had a chat with Farrah Campbell, Chase Douglas, Serhat Can, and Emrah Şamdan about some of our favorite upcoming re:Invent sessions.

re:Invent Guide: Databases
This isn’t exclusively serverless, but there are a few DynamoDB sessions in here that should be super interesting.

Serverless Stories 📖

How Small Startups Move Incredibly Fast With Serverless
Serverless and startups go together like peanut butter and jelly. Dadi Atar recounts why serverless was the right decision for his latest startup journey.

Serverless Big Data: lessons learned building an event ingestion pipeline on Amazon Web Services
Luca Bianchi dives into the details of building a big data ingestion pipeline with serverless.

Story of serverless (Lambda) to move large data from DynamoDB
Always love seeing experiments like this. Bala Dutt and Sachin Maheshwari discuss their experiment that got to processing 7k records per second using DynamoDB and Lambda.

Using CodePipeline to Automate Serverless Applications Deployment
Pedro Lourenço shows you how to build a fully automated CI/CD pipeline with CodePipeline and CodeBuild. He then explains why after using it for six months, his team outgrew it.

Serverless Concepts 🏗

How to access VPC and internet resources from Lambda without paying for a NAT Gateway
Paul Swail points out a workaround for synchronous Lambda workflows needing VPC access and Internet access. Lambdas calling Lambdas, yup, sometimes it just makes sense.

What are Serverless Components?
Ryan Jones has a series of articles on Serverless Components from the makers of the Serverless Framework. These are definitely worth knowing about. Also see: What are the benefits of Serverless Components? and How to use Serverless Components?

Take the legwork out of API Gateway troubleshooting
This is a great new feature in Serverless Framework Pro. Verne Lindner explains why tracking down API Gateway errors can be hard, and how the new request explorer makes corresponding logs easier to find.

The State of AWS Lambda Supported Languages & Runtimes (Updated November 2019)
AWS has released several new Lambda runtimes, Fernando Medina Corey gets you up to speed on all the latest ones.

Data Analytics with Serverless Services
Emrah Samdan outlines the pros and cons of building a serverless analytics pipeline.

Serverless Patterns implemented: using an SQS queue as a DLQ for a SNS topic
Vicenç García has implemented another pattern from my serverless patterns post. This time he tackles the Notifier pattern with the added benefit of the recently released SNS DLQs.

Serverless Tutorials 🏗

Implement the new Stripe checkout workflow with a React Amplify project (serverless and SCA compliant)
Straightforward use case for creating a simple checkout workflow using Stripe and a serverless backend.

Building a serverless data pipeline using Kinesis > Lambda > S3 Glacier pt.2
Part 2 of Serkan Özal’s series covering how to build a serverless data pipeline with Kinesis, Lambda, and AWS S3 Glacier.

Using the AWS CDK to build scheduled Lambda Functions
Maarten Thoelen provides a complete sample for creating Lambda functions and building a CI/CD pipeline using infrastructure as code.

How to build serverless cursor-based pagination service on the AWS Cloud
Andy Nguyen shows an alternate technique to pagination. If you’re using DynamoDB as your database, then this method would be your main option.

How to deploy a Lambda@Edge function with AWS CloudFormation
This is an interesting use case that merges audio files using Lambda@Edge to deliver audio responses to Alexa. Stephane Couzinier explains the whole process (including a CI/CD pipeline) and provides the finished code.

Serverless Security 🔒

How to build secure serverless applications for PCI-DSS compliance?
Thinking about storing data that requires PCI-DSS compliance? My advice: DON’T. But if you need to, this article from Ryan Jones will get you asking the right questions.

Serverless Reads 🤓

New Lambda Features – Pre-re:Invent – Nov. 2019
The number of new serverless announcements are overwhelming. Fernando Medina Corey picks out a few of them and gives us more detail on why they are important.

Is Your CI/CD Pipeline Ready For Chaos?
John Gilbert asks some important questions about your CI/CD process.

The top 30 Amazon products and services tech pros used this year
2nd Watch looked at their customer’s usage of AWS products, and there are some interesting ones at the top, including DynamoDB, SNS, and even Lambda, with 83% of their customer using it.

7 of the Best Benefits of AWS Lambda for Your Cloud Computing System
Trisha Paine outlines seven benefits to AWS Lambda. Hard for me to argue with these.

When you have time during your commute… 🚙

Episode #24: Serverless Application Security with Ory Segal (Part 2)
In this episode, I continue my conversation with Ory Segal about Serverless Application Security. We finish reviewing the CSA Top 12 Most Critical Risks for Serverless Applications, and discuss the future of security for serverless and ephemeral compute.

When the teams at AWS can’t even wait until re:Invent… 🤷‍♂️

Once again, AWS has flooded us with a plethora of announcements regarding new features and products. I haven’t had the time to read through all of these, let alone actually digest and process what they mean. I’ve included 54 links here to the announcements and related blog posts, and there are even more on the What’s New blog! I’m listing all of the ones that seemed interesting and relevant to serverless, and categorized them the best I could. It’s going to take us awhile to make sense of all these, so stay tuned for more in-depth analysis in the weeks to come.

Event-Driven Workflows:

• AWS Lambda Supports Destinations for Asynchronous Invocations (this ones HUGE!)
• AWS Lambda Supports Failure-Handling Features for Kinesis and DynamoDB Event Sources
• ECR events now published to EventBridge
• EventBridge Support in Amazon Elastic Container Registry
• Amazon Kinesis Producer Library (KPL) now supports ListShards API for efficient scaling of producer applications

Observability and Monitoring:

• Announcing Amazon CloudWatch ServiceLens
• AWS X-Ray offers improved trace analysis and identification of service disruption
• Introducing Amazon CloudWatch Synthetics – Now in Preview
• Amazon CloudWatch Now Includes Contributor Insights – in Preview
• Visualize and Monitor Highly Distributed Applications with Amazon CloudWatch ServiceLens
• Amazon Athena adds four new query-related metrics
• New Amazon CloudWatch Contributor Insights for Amazon DynamoDB (Preview) helps you identify frequently accessed keys and database traffic trends

Database:

• Amazon Athena Adds support for User Defined Functions (UDF)
• Amazon Athena adds support for running SQL queries across relational, non-relational, object, and custom data sources
• You now can enhance availability of your existing Amazon DynamoDB tables by adding global replicas
• New – Convert Your Single-Region Amazon DynamoDB Tables to Global Tables
• AWS DataSync adds the ability to schedule data transfers
• Use Apache Hive Metastore as a metadata catalog with Amazon Athena (Preview)
• AWS AppSync adds server-side Caching and DynamoDB transactions support for GraphQL APIs

Developer Tools:

• AWS SAM CLI simplifies deploying serverless applications with single-command deploy
• AWS CodeBuild Adds Support for Test Reporting (Beta)
• Simplify application configuration with AWS AppConfig
• Amplify Console and CLI now work together
• AWS Cloud Development Kit (CDK) – Java and .NET are Now Generally Available
• Access your AWS Regions faster using the AWS Management Console
• Safe Deployment of Application Configuration Settings With AWS AppConfig
• Announcing Cloud Debugging (beta) for Debugging Your Applications Running in the Cloud with JetBrains IDE
• Amplify Console now provides visibility into backend environments provisioned by the Amplify CLI

Machine Learning:

• Amazon Comprehend launches real time custom classification
• Amazon Athena adds support for invoking machine learning models in SQL queries
• New for Amazon Aurora – Use Machine Learning Directly From Your Databases
• Amplify CLI announces new GraphQL transform feature for orchestrating multiple AI/ML use cases
• Now get additional details/metrics around all your algorithm runs with AutoML
• How AWS is putting machine learning in the hands of every developer and BI analyst
• Amazon Lex Now Supports Sentiment Analysis

Security:

• Announcing AWS Managed Rules for AWS WAF
• AWS CloudTrail announces CloudTrail Insights
• Amazon Cognito now supports Sign in with Apple
• Access resources within your Amazon Virtual Private Cloud using Amazon Kinesis Data Analytics
• Amazon Cognito now supports account recovery method prioritization
• Identify unused IAM roles easily and remove them confidently by using the last used timestamp
• Encrypt your Amazon DynamoDB data by using your own encryption keys
• Amazon Kinesis Data Firehose Adds Support For Customer-Provided Keys for Server-Side Encryption
• AWS launches Tag Policies
• New – Use Tag Policies to Manage Tags Across Multiple AWS Accounts
• Use IAM to share your AWS resources with groups of AWS accounts in AWS Organizations

General:

• S3 Replication Update: Replication SLA, Metrics, and Events
• Amazon S3 Replication Time Control for predictable replication time, backed by an SLA
• Amazon CloudFront announces 10 new Edge locations including its first Edge location in Rome, Italy
• Amazon Transcribe now Supports Alternative Transcriptions
• Amazon SES Announces Account-Level Suppression List
• AWS Serverless Application Repository Adds Verified Author Badges for Application Publishers
• Amazon Transcribe Now Supports Speech-to-text in 7 Additional Languages

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

December 2-6, 2019 – AWS re:Invent 2019

December 3, 2019 – Taco Tuesday with the AWS Heroes

December 4, 2019 – #ServerlessForEveryone Community Party @ AWS re:Invent 2019

December 14, 2019 – ServerlessDays Fukuoka Japan

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Ryan Kroonenburg (@KroonenburgRyan). Ryan is the co-founder of A Cloud Guru and an AWS community hero (you might remember when I featured his brother Sam a few weeks ago). When Ryan was a solution architect five years ago, he taught himself AWS, and realized the community was underserved in AWS courses. Fast forward to today, and Ryan’s work with A Cloud Guru has reached over one million students and he continues to teach the community about AWS, serverless, and the cloud. Thank you for making serverless accessible to all who want to learn, Ryan! 🙌

Final Thoughts 🤔

I don’t know about you, but my brain is fried from trying to process all these new AWS serverless announcements, and we’re still over a week away from the big keynotes at re:Invent. I have so many things I want to write about and share, but with all this information to look at, there just isn’t enough time in the day. And if you’re feeling overwhelmed, don’t sweat it. I have a feeling that we’ll be seeing some excellent posts, videos, and tutorials from the serverless community over the next few months to help us make sense of it all.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Hoping to see you at re:Invent,
Jeremy