Npm Go Aws project - Feb 2020
A new project for a rainy Sunday
At work we are reviewing Cats and Monix, so as pragmatic relief I thought I would embrace what I think is really the simple serverless future, and record how and what.
Again, all notes are for myself, so next time I have to do this I have crib notes.
Good reference at serverless-stack.com
And in case you might miss it Amplify, which I did not use since I’m being a GoLang lambda dev: https://aws-amplify.github.io/docs/
tools
- Windows 10 surface pro.
- Notepad++ for all ad-hoc scripting etc
-
Microsoft OneNote for all notes and so on, excellent searching.
- Npm and tools for the npm site
- amplify for AWS
- GoLand IDE for the reactjs and GoLang editing
- Go language for the serverless functions, 1.13.6
- AWS for the cloud native
- GitHub for the repos
-
GitBash for launching GoLand, and for Git.
- An old Jekyll for this documentation
- Atom editor to edit the markdown in this blog - this is historic, and its the only thing I use Atom for so it keeps the blog separate from my many other projects.
- Filezilla for blog upload
Editing the blog
Open up Atom and load c:\all_dev\websites... Open cmd and cd to the same base dir.
bundle exec jekyll serve
This is encapsulated in the startJekyll.bat. Then open a browser at http://127.0.0.1:4000/
Creating the npm project
read the manual. But in summary
cd c:\all_dev\blackseas
npx create-react-app npm-black-seas
cd npm-black-seas
npm start
In this case I post created the github repo, below I do it first for the golang part.
golang
I set the following, but I am going to code in modules.
GOPATH=C:\all_dev\go-work
GOROOT=C:\tools\go
Creating the GoLang project
As ever, read the documentation
I started by creating my private repo in github and cloning it.
cd c:\all_dev\blackseas
git clone https://github.com/user/go-black-seas
cd go-black-seas
go mod init github.com/user/go-black-seas
Since I am going to have many functions, all in the same repo, I’m going to then drag stuff about a bit. Plan is to have loads of modules under jgibbons.com, so I end up with modules, such as bsmap in this kind of path:
C:\all_dev\blackseas\go-black-seas\src\jgibbons.com\bsmap
I them move the go.mod file into that dir and change it to read
module jgibbons.com/bsmap
go 1.13
Each AWS function will basically have its own project under there. Within each function I am going to have several entry points. One for the AWS API entry point with the boiler plan in, and one for local rest running.
bsmap\
aws\ The AWS boiler plate which will invoke the private contents.
rest\ The end point I can run locally without the AWS api stuff.
private\ The functionality
Opening the IDE
Git and GoLand were not playing nicely
GoLand isn’t happy with my local Git, so to avoid googling and hassle, I open up git bash and start the IDE from in there. So in GitBash home dir I have goLand.sh :
"c:\Program Files\JetBrains\GoLand 2019.1.3\bin\goland64.exe"
Which I run with
./goLand.sh
AWS and GoLang getting started
GoLang AWS boilerplate
I have no idea why they want me to jump like this, but here are the links first:
AWS api proxy integration for lambdas
The best blog on Go for AWS lambdas I found AWS docs on lambdas
I will not paste in the boilerplate as Alex Edwards blog above was what got me going, so please support him!
AWS components
No one discusses naming conventions much, so I will go with
- S3 bucket: lowercase with dashes not spaces, eg this-is-a-bucket
- Function Name: Camel Case, eg ThisIsAFunction
- API gateway API name: Default adds -api to the end, eg ThisIsAFunction-API
- Api gateway path: /lowercase, eg /thisisafunction
S3 bucket
AWS how to load data from S3 bucket
API gateway
Also worth reading up on CORS, as running npm locally to call into AWS is nice.
Lambda
Remember to publish it to a stage! Choose a preprod name such as beta.
eg For windows:
go.exe get -u github.com/aws/aws-lambda-go/cmd/build-lambda-zip
cd c:/all_dev/vanguard/go-van-warbanddata/functions/helloworld
set GOOS=linux
go build -o hello hello.go
build-lambda-zip.exe -o hello.zip hello
This builds you a zip file you can drag into the aws console.
Lambda and cors and AWS Api gateway
OK, took me ages of random hacking, so in summary. Add this to your lambda, and don’t bother fiddling within the APIGateway cors config. The APIGateway cors settings are supposed to add headers to the reply, but if you are doing slightly more complex Lambdas then you need to add this:
resp:= events.APIGatewayProxyResponse{
StatusCode: http.StatusOK,
Headers: corsHeaders(),
Body: string(bytes),
IsBase64Encoded: false,
}
return resp, nil
}
func corsHeaders() map[string]string{
headers := make(map[string]string)
headers["Access-Control-Allow-Origin"] = "*"
return headers
}
My project
The project is black-seas. So the things I need are in region eu-west-1.
s3 bucket called motley-black-seas No public access or versioning.
AWS Lambda function called BlackSeasMap
AWS IAM role called role-black-seas-microservice which I created when using the console to create the Lambda, selecting create new role from AWS policy templates. Policy selected was Simple microservice permissions.
An APIGateway, HTTP API (Beta) called BlackSeasMap-Api, created two stages, beta and prod. The initial path was /BlackSeasMap
c:\all_dev\blackseas\go-black-seas GoLang project, also in GitHub, with a go.mod in director src\jgibbons.com\bsmap. The go.mod looks like this:
module jgibbons.com/bsmap
go 1.13
require (
github.com/aws/aws-lambda-go v1.13.3
github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect
github.com/stretchr/objx v0.2.0 // indirect
github.com/urfave/cli v1.22.2 // indirect
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
rsc.io/quote v1.5.2
)
There are some imports above which just proved useful on another project..
c:\all_dev\blackseas\npm-black-seas Reactjs project for the UI, also in GitHub
AWS serverless
Personally I am not using this as I am trying to get away from Docker now, its not a required solution when you have cloud native serverless.
A note on hello world lambda
If you create a hello world lambda as below:
package main
import (
"fmt"
"encoding/json"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
"log"
"net/http"
"os"
)
/*
https://www.alexedwards.net/blog/serverless-api-with-go-and-aws-lambda
*/
var errorLogger = log.New(os.Stderr, "ERROR ", log.Llongfile)
type MyEvent struct {
Name string `json:"name"`
}
// Add a helper for handling errors. This logs any error to os.Stderr
// and returns a 500 Internal Server Error response that the AWS API
// Gateway understands.
func serverError(err error) (events.APIGatewayProxyResponse, error) {
errorLogger.Println(err.Error())
return events.APIGatewayProxyResponse{
StatusCode: http.StatusInternalServerError,
Body: http.StatusText(http.StatusInternalServerError),
}, nil
}
// Similarly add a helper for send responses relating to client errors.
func clientError(status int) (events.APIGatewayProxyResponse, error) {
return events.APIGatewayProxyResponse{
StatusCode: status,
Body: http.StatusText(status),
}, nil
}
//func HandleRequest(ctx context.Context, name MyEvent) (string, error) {
func HandleRequest(req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
name,ok:= req.QueryStringParameters["name"]
if (!ok) {
return clientError(http.StatusBadRequest)
}
ret:= fmt.Sprintf("Hello %s!", name )
bytes, err := json.Marshal(ret)
if err != nil {
return serverError(err)
}
// Return a response with a 200 OK status and the JSON book record
// as the body.
return events.APIGatewayProxyResponse{
StatusCode: http.StatusOK,
Body: string(bytes),
}, nil
}
func main() {
lambda.Start(HandleRequest)
}
You can test this, by pretending to be the ApiGateway forwarding it, for eample this Json will send in the name JoJo.
You can do this in the Test button (or dropdown, configure events) in the AWS console.
{
"resource": "Resource path",
"path": "Path parameter",
"httpMethod": "GET",
"headers": {},
"multiValueHeaders": {},
"queryStringParameters": {
"name": "JoJo"
},
"multiValueQueryStringParameters": {},
"pathParameters": {},
"stageVariables": {},
"requestContext": {},
"body": "A JSON string of the request payload.",
"isBase64Encoded": false
}
When you hit test you will get success with the output:
{
"statusCode": 200,
"headers": null,
"multiValueHeaders": null,
"body": "\"Hello JoJo!\""
}
User Authentication
In AWS this is cognito and user pools/identity.
See my next post.