As our code base i getting bigger it’s better if we split our code into different files.
When i started doing this process personally, i faced lot of issues and this took me much more time to understand fully. I will try you take you through the same steps so that we can learn from the same mistakes and get better understanding of flask.
Since we are doing refracting, first step is to setup git so have entire history of our changes. It’s always recommended to use git for any project size.
git init
git remote add origin https://github.com/pythexcel/tutorial.git
before we do commit, we need to setup gitignore file which we can take from here https://github.com/github/gitignore/blob/master/Python.gitignore
git add -A && git commit -m "initial project files"
git push origin master
This is how our code looks at this stage https://github.com/pythexcel/tutorial/commit/4e860510bc26a4438c6474b019500015164814b2
Step1
First let’s try to put our database connection in a separate file db.py
First lets make a new branch in git
git checkout -b db
We will make a new file db.py in the same directory and import it
Look at changes here https://github.com/pythexcel/tutorial/commit/e99f991d510d1534de13292a39da7fdd0620749c
Now if you run the app you get an error
ImportError: attempted relative import with no known parent package
This error means that python is not able to do relative import as it has no package.
To understand the reason for this error go through this blog https://napuzba.com/a/import-error-relative-no-parent/
Basically we need to add a special file named __init__.py and which tells python the our current directly is a package https://stackoverflow.com/questions/448271/what-is-init-py-for
Let’s add the init file https://github.com/pythexcel/tutorial/commit/801d6eef931049b84e6951e70d1cf30c233814a2 and now the error is gone.
At this stage, its important to understand about packages, __init__.py file so you can find many resources online for this.
Now lets actually move the code to db.py. If we move these lines to db.py
app.config["MONGO_URI"] = "mongodb://localhost:27017/todo"
mongo = PyMongo(app)
for this we need the app variable in db.py file and return the mongo variable so let’s see how to do it
https://github.com/pythexcel/tutorial/commit/38755b9c42d769f4a72ab25eabca2d2468195546
As you can see in the code, i moved the code to a separate file and it works well.
P.S. Every time you need to fire an api via postman to actually confirm if it works. When you run the app you won’t see any error, only when you fire the api you will see it.
Step2
Now, lets move the register routes to a different file. Basically i want all routes related to login, registration etc to be moved to a different file.
Let’s make a new file auth.py for it. First problem when we move the code to other file is we need @app variable to register routes. To solve this problem we need to use flask blueprints http://flask.pocoo.org/docs/1.0/tutorial/views/
P.S blueprint is a flask solution to do exactly this i,e structure code, we will ultimately use this only finally but in blog we will go step by step and evolve towards it
This is how is setup the register route with blueprint https://github.com/pythexcel/tutorial/commit/b3146338b7c8834f935f7a4e2aa32a73c724c1a5
Quite simple, just move the code of registration route and relevant imports.
Now if you run the route /auth/register it works!
Note, our route changed to /auth/register instead of /register as we used blueprint with url_prefix as auth
Now, lets move login route as well
https://github.com/pythexcel/tutorial/commit/45653711d3098db3b576d08ddbc7813a8d55daf9
and next profile route
https://github.com/pythexcel/tutorial/commit/d92baff956c2fd7a0c8b3092c9ec52a5977889c4
Hmm, the profile route doesn’t work properly. It didn’t give any error but returns the logged_in user as empty {}
The reason for this because we have users = [] defined on both the files i.e hello.py and auth.py so we have two different array for users which is causing problems. Now we need to resolve this i.e have one global users array.
It’s important to understand this problem properly as this has deep architecture issues.
In our application, we had a users = [] which was global to all requests. Which means all requests manipulate the same global array of users. This should never be done in production apps because the way flask works is that when there are multiple requests, it will open multiple threads and every thread will in parallel try to modify the users array. this will cause lot of problems and inconsistencies. We should always use a database or caching system to store such data never a variable. More details here https://www.reddit.com/r/flask/comments/1azce1/ask_flask_application_wide_variable/
But for the current case to solve the problem we can use app.config this is a application wide variable.
We can solve this problem like this https://github.com/pythexcel/tutorial/commit/cb98df8e18cc3a88619b91c4db6b43d27ce3f8f5 using app.config but this not at all recommended!
Step3
Let’s move all our todo routes to a different file
As soon as we start down this path, first we need be able to pass mongo db connection available in every request.
To do this we use a special variable called “g”
g is a special object that is unique for each request. It is used to store data that might be accessed by multiple functions during the request. The connection is stored and reused instead of creating a new connection if get_db is called a second time in the same request.
It’s important to note here that “g” is only available when we are processing request, not in the global state. To give example of this
If we try to assign “mongo” connection in our init_db function it will give error
RuntimeError: Working outside of application context.
See this code https://github.com/pythexcel/tutorial/commit/8a77341b9310675873f8ab2e690362673b791f73 especially the db.py
Here we tried to access “g” in global context, rather we should do it in each route.
To fix this we need create a new function called get_db() and use “g” there. This function get_db() will called from every route function.
https://github.com/pythexcel/tutorial/commit/df4468c7d54b3d3b464e2eded2e86f998417e337
This is the code now and it works. We added a new object called “current_app”
current_app is another special object that points to the Flask application handling the request. Since you used an application factory, there is no application object when writing the rest of your code. get_db will be called when the application has been created and is handling a request, so current_app can be used.
It’s important to understand what context in flask, i found this blog which was helpful to understand it http://kronosapiens.github.io/blog/2014/08/14/understanding-contexts-in-flask.html
Now this approach works, but there is a problem that for every request we are creating a new mongo connection. This is not good, we should have a global connection which should be reused. As we know from “g” documentation that this is created for every request and destroyed after a request is finished. “g” should be used to pass data between functions but within a single request only.
So we need a different approach to solve this problem.
To solve this problem if we thing logically
- we need to initialize mongo connection globally in our hero.py file like before
- we need to import the mongo connection variable in our route files
https://github.com/pythexcel/tutorial/commit/759466191dd062312110970d90ca8f1535f329c6
If you look at the console in terminal, you will connecting to db called twice but this is only because debug mode.
https://stackoverflow.com/questions/49524270/app-initializes-twice-is-this-typical-behavior
Notice how i imported the variable mongo ” from .hello import mongo”
Step4
We need to enable the admin_required route as well. We will follow similar process as before
https://github.com/pythexcel/tutorial/commit/516eaf621a25d08d9057d2db35470403d767210c
Step5
Let’s move all jwt related stuff to a separate file jwt.py
https://github.com/pythexcel/tutorial/commit/9d7127eeb0f0bbe3eaf2fcdba20ba13bad069292
At this stage our main hello.py file looks like this
from flask import Flask
# def create_app():
app = Flask(__name__)
from . import db
mongo = db.init_db(app)
from . import jwt
jwt = jwt.init_jwt(app)
print("hello py called")
app.config["users"] = []
from . import auth
app.register_blueprint(auth.bp)
from . import todo
app.register_blueprint(todo.bp)
The code looks much better now and much more manageable.
Step6
Let’s make a configuration file. Flask supports reading data from a config.py file, so that we can easily have different configuration for environments. Let’s see how to do it
e.g the mongo connection url, jwt token secret all these should be configurable we should be able to provide different for multiple env like staging, production etc
https://github.com/pythexcel/tutorial/commit/0cf55e900e9e5c6656c3ec8deadc9a8bfdb4dc43
Here we have created a config.py file
This is quite simple, and for doing this on multiple environments see these links
http://flask.pocoo.org/docs/1.0/config/#development-production
https://stackoverflow.com/questions/50610832/managing-configurations-for-different-environments
Step7
Now we will have fully refactored over application, now lets see further some more best practices we should use and specific features of blueprints.
http://flask.pocoo.org/docs/1.0/api/#flask.Blueprint.before_request
Blueprint provides many wrap functions like before_request, after_request etc which we can use for specific set of url and add cross cutting logic to it.
STEP8
One last final step is requirements.txt file.
Basically our all application dependencies need to be captured in a file, so that when ever we transfer the app to a different environment or some one else clones and install it he can straight away install all dependencies
Write this on command line
pip freeze > requirements.txt
this will create a file with all dependencies which you can commit to git and when we want to install all dependencies we can do
pip install -r requirements.txt