Flask met Zappa op AWS Lambda draaien

[Boudewijn]

Hoi allemaal,

Ik heb een vet mooie API gemaakt in flask en die wil ik op AWS lambda deployen. Gewoon omdat ik er meer van wil leren. Ik volg grofweg deze tutorial en heb een pip freeze > requirements.txt gedaan


Nu krijg ik bij wat ik ook doe de volgende resultaten:

code:

(.env) johns-MacBook-Pro:code john$ zappa update dev
Calling update for stage dev..
Downloading and installing dependencies..
 - markupsafe==1.1.1: Downloading
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 27.5k/27.5k [00:00<00:00, 77.7kB/s]
Packaging project as zip.
Uploading zappa-test-dev-1583507021.zip (5.3MiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.57M/5.57M [03:02<00:00, 30.6kB/s]
Updating Lambda function code..
Updating Lambda function configuration..
Uploading zappa-test-dev-template-1583507216.json (1.6KiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.63k/1.63k [00:01<00:00, 1.49kB/s]
Deploying API Gateway..
Scheduling..
Unscheduled zappa-test-dev-zappa-keep-warm-handler.keep_warm_callback.
Scheduled zappa-test-dev-zappa-keep-warm-handler.keep_warm_callback with expression rate(4 minutes)!
Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.
(.env) johns-MacBook-Pro:code john$

Prima, als ik in mijn zappa tail kijk kom ik bij deze error uit:

code:

1583507428944] [DEBUG] 2020-03-06T15:10:28.944Z 9a92f078-af9c-4254-b964-78262a111a25 Zappa Event: {'resource': '/', 'path': '/', 'httpMethod': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Accept-Language': 'nl,en;q=0.9,en-US;q=0.8,de;q=0.7', 'CloudFront-Forwarded-Proto': 'https', 'CloudFront-Is-Desktop-Viewer': 'true', 'CloudFront-Is-Mobile-Viewer': 'false', 'CloudFront-Is-SmartTV-Viewer': 'false', 'CloudFront-Is-Tablet-Viewer': 'false', 'CloudFront-Viewer-Country': 'NO', 'dnt': '1', 'Host': 'uzook515je.execute-api.eu-central-1.amazonaws.com', 'Referer': 'https://eu-central-1.console.aws.amazon.com/apigateway/home?region=eu-central-1', 'sec-fetch-mode': 'navigate', 'sec-fetch-site': 'cross-site', 'sec-fetch-user': '?1', 'upgrade-insecure-requests': '1', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36', 'Via': '2.0 800854209ac3edaec5b9f8615c0bbd59.cloudfront.net (CloudFront)', 'X-Amz-Cf-Id': 'YyISe-jvM0qEahGec90uuKO2xFmWtHJKjL_l1FTJIhZyk7GHyrBJjQ==', 'X-Amzn-Trace-Id': 'Root=1-5e6267e4-e3b8da75179d28bdb16c65be', 'X-Forwarded-For': '82.102.27.251, 52.46.52.85', 'X-Forwarded-Port': '443', 'X-Forwarded-Proto': 'https'}, 'multiValueHeaders': {'Accept': ['text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9'], 'Accept-Encoding': ['gzip, deflate, br'], 'Accept-Language': ['nl,en;q=0.9,en-US;q=0.8,de;q=0.7'], 'CloudFront-Forwarded-Proto': ['https'], 'CloudFront-Is-Desktop-Viewer': ['true'], 'CloudFront-Is-Mobile-Viewer': ['false'], 'CloudFront-Is-SmartTV-Viewer': ['false'], 'CloudFront-Is-Tablet-Viewer': ['false'], 'CloudFront-Viewer-Country': ['NO'], 'dnt': ['1'], 'Host': ['uzook515je.execute-api.eu-central-1.amazonaws.com'], 'Referer': ['https://eu-central-1.console.aws.amazon.com/apigateway/home?region=eu-central-1'], 'sec-fetch-mode': ['navigate'], 'sec-fetch-site': ['cross-site'], 'sec-fetch-user': ['?1'], 'upgrade-insecure-requests': ['1'], 'User-Agent': ['Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36'], 'Via': ['2.0 800854209ac3edaec5b9f8615c0bbd59.cloudfront.net (CloudFront)'], 'X-Amz-Cf-Id': ['YyISe-jvM0qEahGec90uuKO2xFmWtHJKjL_l1FTJIhZyk7GHyrBJjQ=='], 'X-Amzn-Trace-Id': ['Root=1-5e6267e4-e3b8da75179d28bdb16c65be'], 'X-Forwarded-For': ['82.102.27.251, 52.46.52.85'], 'X-Forwarded-Port': ['443'], 'X-Forwarded-Proto': ['https']}, 'queryStringParameters': None, 'multiValueQueryStringParameters': None, 'pathParameters': None, 'stageVariables': None, 'requestContext': {'resourceId': 'bt4oh3xv8i', 'resourcePath': '/', 'httpMethod': 'GET', 'extendedRequestId': 'I-UrxF83liAFpGw=', 'requestTime': '06/Mar/2020:15:10:28 +0000', 'path': '/dev', 'accountId': '037957353425', 'protocol': 'HTTP/1.1', 'stage': 'dev', 'domainPrefix': 'uzook515je', 'requestTimeEpoch': 1583507428871, 'requestId': 'd2a31363-6436-4ee5-b703-27c9aa019082', 'identity': {'cognitoIdentityPoolId': None, 'accountId': None, 'cognitoIdentityId': None, 'caller': None, 'sourceIp': '82.102.27.251', 'principalOrgId': None, 'accessKey': None, 'cognitoAuthenticationType': None, 'cognitoAuthenticationProvider': None, 'userArn': None, 'userAgent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36', 'user': None}, 'domainName': 'uzook515je.execute-api.eu-central-1.amazonaws.com', 'apiId': 'uzook515je'}, 'body': None, 'isBase64Encoded': False}
[1583507428944] [DEBUG] 2020-03-06T15:10:28.944Z 9a92f078-af9c-4254-b964-78262a111a25 host found: [uzook515je.execute-api.eu-central-1.amazonaws.com]
[1583507428944] [DEBUG] 2020-03-06T15:10:28.944Z 9a92f078-af9c-4254-b964-78262a111a25 amazonaws found in host
[1583507428944] 'NoneType' object is not callable

Prima ,dat lijkt er dus op dat mijn app.py gewoon niet aan te roepen is... in de shell met Python3.7 lukt het prima. python3 app.py draait keurig.

Python:

from flask import Flask, jsonify
from flask_restful import Api
from flask_jwt import JWT

from security import authenticate,identity
from resources.user import UserRegister
from resources.question import Question, QuestionList
from resources.answer import Answer, AnswerList
from resources.category import Category, CategoryList
from resources.quizsession import QuizSession
from resources.attachment import Attachment, AttachmentList
from models.attachment import AttachmentModel

from  datetime import timedelta
from db import db
from pathlib import Path

from secrets import db_secrets as db_secrets


app = Flask(__name__)
#app.config['SQLALCHEMY_DATABASE_URI'] = "sqlite:///data.db"
app.config['SQLALCHEMY_DATABASE_URI'] = "mysql+mysqldb://"+db_secrets["username"]+":"+db_secrets["password"]+"@"+db_secrets["host"]+":3306/"+db_secrets["databasename"]


#dit zet ALLEEN De flask SQLAlchemy modificatie tracker uit. niet die van SQLAlchemy zelf!
app.config['SQLALCHEMY_TRACK_MODIFICATIONS']=False

app.secret_key = 'KNIP

'
app.config['JWT_AUTH_URL_RULE'] = '/auth'
app.config['JWT_EXPIRATION_DELTA'] = timedelta(seconds=1800)
app.config['JWT_AUTH_USERNAME_KEY'] = 'username'
# https://blog.tecladocode.com/learn-python-advanced-configuration-of-flask-jwt/


api = Api(app)


@app.before_first_request
def create_tables():
    db.create_all()


@app.before_first_request
def create_resource_dir():
    # 1200 x 630

    for path in [AttachmentModel.FILE_DIR, AttachmentModel.FILE_DIR+'/original']:
        try:
            Path(path).mkdir(parents=True, exist_ok=True)
        except OSError:
            print ("Creation of the directory %s failed" % path)
            exit(1)
    return


jwt = JWT(app, authenticate, identity)

api.add_resource(Category, '/category/<string:name>')
<nog wat meer resources>



@app.route('/')
def hello_world():
    return jsonify({"message": "Hello World!"})



@jwt.auth_response_handler
def customized_response_handler(access_token, identity):
    return jsonify({
        'access_token': access_token.decode('utf-8'),
        'user_id': identity.id
    })


if __name__ == '__main__':
    db.init_app(app)
    app.run(debug=True, port=5000)

Ziet er volgens mij allemaal prima uit.


Punt is dat ik niet helemaal snap waarom dit wel uit wil voeren lokaal en niet in de lambda-context. Ja ik snap dat voor elk request dan een instance wordt gestart en de boel in principe stateless is, maar waarom dit nu niet wil werken...


Wat heb ik geprobeerd?

- een simpele lege app werkt wel. Dit werkt prima:

code:

johns-MacBook-Pro:flask-app john$ cat zappa_settings.json
{
    "dev": {
        "app_function": "app.app",
        "profile_name": "default+",
        "project_name": "flask-app",
        "runtime": "python3.7",
        "s3_bucket": "zappa-i7n7qvbbl"
    }
}johns-MacBook-Pro:flask-app john$ cat app.py
from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/')
def hello_world():
    return jsonify({"message": "Hello World!"})

if __name__ == '__main__':
 app.run()

Gelezen:
- https://github.com/Miserlou/Zappa/issues/1396
- ik heb de db.init_app(app) uitgezet (ja onwenselijk), dit wordt hier (https://github.com/Miserlou/Zappa/issues/1090) op 11 september 2017 gesuggereerd
- Ik heb de zappa init gedraaid in de root van mijn project, waar ook app.py staat . En, een niveau hoger in het fileystem.

Mijn zappa_settings.json:

code:

(venv) johns-MacBook-Pro:code john$ cat zappa_settings.json
{
    "dev": {
        "app_function": "app.app",
        "profile_name": "default+",
        "project_name": "zappa-test",
        "runtime": "python3.7",
        "s3_bucket": "<S3 Bucket naam>"
    },
    "dev_ap_east_1": {
        "aws_region": "ap-east-1",
        "extends": "dev"
    },

En nog meer regions.

Een niveau hoger is de app_function code.app.app en dat werkt ook niet.

- ik heb mijn app.py gerenamed. Mogelijk gaat hier iets mis. Uiteraard netjes zappa update dev hierna gedraaid. Dit mag ook niet baten.


Zelfs met een bijna helemaal uitgeklede app werkt het niet:

Python:

(.env) johns-MacBook-Pro:code john$ grep -v "^#" app.py
from flask import Flask, jsonify
from flask_restful import Api
from flask_jwt import JWT

from security import authenticate,identity
from resources.user import UserRegister
from resources.question import Question, QuestionList
from resources.answer import Answer, AnswerList
from resources.category import Category, CategoryList
from resources.quizsession import QuizSession
from resources.attachment import Attachment, AttachmentList
from models.attachment import AttachmentModel

from  datetime import timedelta
from db import db
from pathlib import Path

from secrets import db_secrets as db_secrets


app = Flask(__name__)


app.config['SQLALCHEMY_TRACK_MODIFICATIONS']=False

app.secret_key = 'mBEdKy3K8OdoXsX0vqzaJML0qOzgKzey5eMTykKyqE4Q0o9ZxvcGnYhyiXepsvRf'
app.config['JWT_AUTH_URL_RULE'] = '/auth'
app.config['JWT_EXPIRATION_DELTA'] = timedelta(seconds=1800)
app.config['JWT_AUTH_USERNAME_KEY'] = 'username'


api = Api(app)

if __name__ == '__main__':
    app.run()

Frustratie tot en met dus .

[ Voor 100% gewijzigd door Boudewijn op 06-03-2020 16:52 ]

[Boudewijn]

Goed, het werkt ondertussen. Ik ben blokken code uit gaan zetten, en weer aan...

Python:

from flask import Flask, request, jsonify
from flask_restful_swagger import swagger

from flask_restful import Api, reqparse
from resources.user import UserRegister
from resources.question import Question, QuestionList
from resources.answer import Answer, AnswerList
from resources.category import Category, CategoryList
from resources.quizsession import QuizSession, QuizSessionList
from resources.attachment import Attachment, AttachmentList
from models.attachment import AttachmentModel

from  datetime import timedelta
from db import db
from pathlib import Path
from secrets import *

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = "*knip*"


#dit zet ALLEEN De flask SQLAlchemy modificatie tracker uit. niet die van SQLAlchemy zelf!
app.config['SQLALCHEMY_TRACK_MODIFICATIONS']=False

app.secret_key = 'mBEdKy3K8OdoXsX0vqzaJML0qOzgKzey5eMTykKyqE4Q0o9ZxvcGnYhyiXepsvRf'
app.config['JWT_AUTH_URL_RULE'] = '/auth'
app.config['JWT_EXPIRATION_DELTA'] = timedelta(seconds=1800)
app.config['JWT_AUTH_USERNAME_KEY'] = 'username'
# https://blog.tecladocode.com/learn-python-advanced-configuration-of-flask-jwt/


api = swagger.docs(Api(app), apiVersion='0.1')


api.add_resource(Category, '/category/<string:name>')
# en nog meer resources toegevoegd


if __name__ == '__main__':
    db.init_app(app)
    app.run()

Nou ja prima dus, hij draait. Nu kijken hoe ik dit slim ga benutten.