A new project for a rainy Sunday

At work we are reviewing Cats and Monix, so as pragmatic relief I thought I would embrace what I think is really the simple serverless future, and record how and what.

Again, all notes are for myself, so next time I have to do this I have crib notes.

Good reference at serverless-stack.com

And in case you might miss it Amplify, which I did not use since I’m being a GoLang lambda dev: https://aws-amplify.github.io/docs/

tools

Windows 10 surface pro.
Notepad++ for all ad-hoc scripting etc
Microsoft OneNote for all notes and so on, excellent searching.
Npm and tools for the npm site
amplify for AWS
GoLand IDE for the reactjs and GoLang editing
Go language for the serverless functions, 1.13.6
AWS for the cloud native
GitHub for the repos
GitBash for launching GoLand, and for Git.
An old Jekyll for this documentation
Atom editor to edit the markdown in this blog - this is historic, and its the only thing I use Atom for so it keeps the blog separate from my many other projects.
Filezilla for blog upload

Editing the blog

Open up Atom and load c:\all_dev\websites... Open cmd and cd to the same base dir.

bundle exec jekyll serve

This is encapsulated in the startJekyll.bat. Then open a browser at http://127.0.0.1:4000/

Creating the npm project

read the manual. But in summary

cd c:\all_dev\blackseas
npx create-react-app npm-black-seas
cd  npm-black-seas
npm start

In this case I post created the github repo, below I do it first for the golang part.

golang

I set the following, but I am going to code in modules.

GOPATH=C:\all_dev\go-work
GOROOT=C:\tools\go

Creating the GoLang project

As ever, read the documentation

I started by creating my private repo in github and cloning it.

cd c:\all_dev\blackseas
git clone https://github.com/user/go-black-seas
cd go-black-seas
go mod init github.com/user/go-black-seas

Since I am going to have many functions, all in the same repo, I’m going to then drag stuff about a bit. Plan is to have loads of modules under jgibbons.com, so I end up with modules, such as bsmap in this kind of path:

C:\all_dev\blackseas\go-black-seas\src\jgibbons.com\bsmap

I them move the go.mod file into that dir and change it to read

module jgibbons.com/bsmap

go 1.13

Each AWS function will basically have its own project under there. Within each function I am going to have several entry points. One for the AWS API entry point with the boiler plan in, and one for local rest running.

bsmap\
  aws\         The AWS boiler plate which will invoke the private contents.
  rest\        The end point I can run locally without the AWS api stuff.
  private\     The functionality

Opening the IDE

Git and GoLand were not playing nicely

GoLand isn’t happy with my local Git, so to avoid googling and hassle, I open up git bash and start the IDE from in there. So in GitBash home dir I have goLand.sh :

"c:\Program Files\JetBrains\GoLand 2019.1.3\bin\goland64.exe"

Which I run with

./goLand.sh

AWS and GoLang getting started

Getting started

GoLang AWS boilerplate

I have no idea why they want me to jump like this, but here are the links first:

AWS api proxy integration for lambdas

Lambda proxy integrations

Lambda input format

Lambda output format

The best blog on Go for AWS lambdas I found AWS docs on lambdas

AWS hello world

I will not paste in the boilerplate as Alex Edwards blog above was what got me going, so please support him!

AWS components

No one discusses naming conventions much, so I will go with

S3 bucket: lowercase with dashes not spaces, eg this-is-a-bucket
Function Name: Camel Case, eg ThisIsAFunction
API gateway API name: Default adds -api to the end, eg ThisIsAFunction-API
Api gateway path: /lowercase, eg /thisisafunction

S3 bucket

AWS how to load data from S3 bucket

API gateway

AWS API dev guide

Also worth reading up on CORS, as running npm locally to call into AWS is nice.

Lambda

Remember to publish it to a stage! Choose a preprod name such as beta.

AWS how to publish

eg For windows:

go.exe get -u github.com/aws/aws-lambda-go/cmd/build-lambda-zip

cd c:/all_dev/vanguard/go-van-warbanddata/functions/helloworld
set GOOS=linux
go build -o hello hello.go
build-lambda-zip.exe -o hello.zip hello

This builds you a zip file you can drag into the aws console.

Lambda and cors and AWS Api gateway

OK, took me ages of random hacking, so in summary. Add this to your lambda, and don’t bother fiddling within the APIGateway cors config. The APIGateway cors settings are supposed to add headers to the reply, but if you are doing slightly more complex Lambdas then you need to add this:

  resp:= events.APIGatewayProxyResponse{
    StatusCode:        http.StatusOK,
    Headers:           corsHeaders(),
    Body:              string(bytes),
    IsBase64Encoded:   false,
  }

  return resp, nil
}
func corsHeaders() map[string]string{
	headers :=  make(map[string]string)
	headers["Access-Control-Allow-Origin"] = "*"
	return headers
}

My project

The project is black-seas. So the things I need are in region eu-west-1.

s3 bucket called motley-black-seas No public access or versioning.

AWS Lambda function called BlackSeasMap

AWS IAM role called role-black-seas-microservice which I created when using the console to create the Lambda, selecting create new role from AWS policy templates. Policy selected was Simple microservice permissions.

An APIGateway, HTTP API (Beta) called BlackSeasMap-Api, created two stages, beta and prod. The initial path was /BlackSeasMap

c:\all_dev\blackseas\go-black-seas GoLang project, also in GitHub, with a go.mod in director src\jgibbons.com\bsmap. The go.mod looks like this:

module jgibbons.com/bsmap

go 1.13

require (
	github.com/aws/aws-lambda-go v1.13.3
	github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect
	github.com/stretchr/objx v0.2.0 // indirect
	github.com/urfave/cli v1.22.2 // indirect
	gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
	rsc.io/quote v1.5.2
)

There are some imports above which just proved useful on another project..

c:\all_dev\blackseas\npm-black-seas Reactjs project for the UI, also in GitHub

AWS serverless

AWS SAM

Personally I am not using this as I am trying to get away from Docker now, its not a required solution when you have cloud native serverless.

A note on hello world lambda

If you create a hello world lambda as below:

package main

import (
	"fmt"
	"encoding/json"
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
	"log"
	"net/http"
	"os"
)

/*
https://www.alexedwards.net/blog/serverless-api-with-go-and-aws-lambda
 */
var errorLogger = log.New(os.Stderr, "ERROR ", log.Llongfile)

type MyEvent struct {
	Name string `json:"name"`
}

// Add a helper for handling errors. This logs any error to os.Stderr
// and returns a 500 Internal Server Error response that the AWS API
// Gateway understands.
func serverError(err error) (events.APIGatewayProxyResponse, error) {
	errorLogger.Println(err.Error())

	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusInternalServerError,
		Body:       http.StatusText(http.StatusInternalServerError),
	}, nil
}

// Similarly add a helper for send responses relating to client errors.
func clientError(status int) (events.APIGatewayProxyResponse, error) {
	return events.APIGatewayProxyResponse{
		StatusCode: status,
		Body:       http.StatusText(status),
	}, nil
}

//func HandleRequest(ctx context.Context, name MyEvent) (string, error) {
func HandleRequest(req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	name,ok:= req.QueryStringParameters["name"]
	if (!ok) {
		return clientError(http.StatusBadRequest)
	}

	ret:=  fmt.Sprintf("Hello %s!", name )
	bytes, err := json.Marshal(ret)
	if err != nil {
		return serverError(err)
	}

	// Return a response with a 200 OK status and the JSON book record
	// as the body.
	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusOK,
		Body:       string(bytes),
	}, nil
}

func main() {
	lambda.Start(HandleRequest)
}

You can test this, by pretending to be the ApiGateway forwarding it, for eample this Json will send in the name JoJo.

You can do this in the Test button (or dropdown, configure events) in the AWS console.

{
    "resource": "Resource path",
    "path": "Path parameter",
    "httpMethod": "GET",
    "headers": {},
    "multiValueHeaders": {},
    "queryStringParameters": {
        "name": "JoJo"
    },
    "multiValueQueryStringParameters": {},
    "pathParameters":  {},
    "stageVariables": {},
    "requestContext": {},
    "body": "A JSON string of the request payload.",
    "isBase64Encoded": false
}

When you hit test you will get success with the output:

{
  "statusCode": 200,
  "headers": null,
  "multiValueHeaders": null,
  "body": "\"Hello JoJo!\""
}

User Authentication

In AWS this is cognito and user pools/identity.

See my next post.