Creating a re-usable Django app - Part 1

Published byGoogle on

Hi everyone and welcome back for this next tutorial of the Babbler series. This tutorial will cover 2 ways of creating a re-usable Django app.

The first and simplest one is apps for flexible and personal re-use.

The second one is a somewhat less simple way of doing it but which allows re-distribution of your application and that can be a very good thing too!

For the first part of this tutorial we will be using the custom user model, built during the previous tutorial, as a starting point. The second part will be using the theme application built during the Using your own flavour of bootstrap with django tutorial.

Before getting strated: testing

Before we get started on making our app re-usable, we have to make sure our app is working well. To do that we need to write some tests.
Tests are not the subject of this tutorial per say, so we will not be writting them for now. We will be covering test writing in another tutorial.

Re-usable app for flexible personal use Vs redistributable app

What are the main differences and when to go one way or another? Well a redistributable app is, IMHO, an app that you can take as-is and integrate into any project with as few configuration as possible. While a re-usable personal app is more of a base that you know you will need for several project but with some changes from one project to another.

A perfect example of a personal re-usable app is the custom user model. Writing a custom user model is something you will have to do for many projects but never exactly the same way, once you will need a receive_newsletter field, while the next time, you will have to add a country field and so on.
So the basics of inheriting from Django's base models is what you are looking to re-use, then you will be able to focus on the real value-adding process of customizing the user model to the project's needs.

Re-usable app for flexible personal use: How to do it?

The first thing to do is to make sure that everything related to your app is actually included inside your app. Everything related to your app is:

  • models, forms, views, tests and admin modules:
    No sweat there, it's already in your app directory.
  • urls:
    Usually those are located in the main urls.py file and need to be moved out.
  • templates:
    So far our specialized templates have always been built inside our app directory, so we don't need to move anything but you might need to check this for other apps you whish to turn into flexible re-usable apps.
    Also note that a template usually extends a base template and overwrites some named block(s) like content, main, extrahead, footerjs or many others. Those block names are usually "personnal" and will vary from project to project. Usually you will want to use the same name in every single one of your projects but working for different outside projects you may need to adapt to a customer's own naming scheme.
  • static files:
    Any static file which is specifically used only by your app will have to be moved inside the app's own directory. This may include javascripts, css, image and some other files. In our case we will be considering our app to be re-usable for any bootstrap project so we will leave bootstrap's files where they are and other than that we don't have any specific static file so there will be nothing to do.
    If you do have static files to move, use the same naming scheme as with templates (myapp/static/myapp/)
  • other Django classes:
    We don't have any either in our example but you also want to make sure specific Middlewares and ContextProcessors are located inside your app's directory.
  • requirements:
    Build a specific requirements.txt file inside your application with all its dependencies which you will remove from the main requirements.txt file (if they are not needed by any other project specific application)

Now to be managing our app we will use a new repository. The default (or master) branch will hold anything non-project specifc, any project specific field or other functionality will be moved to its separate named branch. If you have several projects sharing a particular caracteristic, feel free to have a branch dedicated to that caracteristic and have "sub-branches" of that branch for each project (ie: the newsletter branch will have everything related to the receive_newsletter field and branches project_a and project_b will both branch off of the newsletter branch while project_c which doesn't require a newsletter will branch off of the default branch).

So let's go ahead with creating the new repository and copying the application inside it (copy the application, do not remove the original files yet).

$ hg init myauth
$ cp -r babbler/myauth/* myauth/

Do not commi yet, we have some cleaning-up to do before the first commit!
First, make sure all the points from the previous check-list have been done, in our case, it means that inside our new repository we should have a urls.py file which looks like this:

from django.conf.urls import patterns, include, url                                       
                                                                                          
from myauth.views import LoginView, LogoutView, RegisterView, EmailSentView, ActivationView, LostPasswordView, LostPasswordEmailSentView, LostPasswordChangeView                    

urlpatterns = patterns('',                                                                
    url(r'^login\.html$', LoginView.as_view(), name='login'),                             
    url(r'^logout\.html$', LogoutView.as_view(), name='logout'),                          
    url(r'^register\.html$', RegisterView.as_view(), name='register'),                    
    url(r'^registration_email_sent\.html', EmailSentView.as_view(), name='confirmation_mail_sent'),
    url(r'^user_activation\.html', ActivationView.as_view(), name='activation'),
    url(r'^lost_password\.html', LostPasswordView.as_view(), name='lost_password'),       
    url(r'^lost_password_mail_sent\.html', LostPasswordEmailSentView.as_view(), name='lost_password_mail_sent'),                                                                    
    url(r'^reset_password\.html', LostPasswordChangeView.as_view(), name='reset_password'),    
) 

After which we have to remove anything project-specific from our application. In our example, any reference to the receive_newsletter field. It's always a good idea to check every single file but there is a small command which can help you work faster:

$ cd myauth
$ grep -rl receive_newsletter *
admin.py
forms.py
migrations/0003_auto__del_field_user_last_name__del_field_user_first_name__add_field_u.py
migrations/0002_auto__add_unique_user_email.py
migrations/0005_auto__add_field_registration_type.py
migrations/0006_auto__chg_field_registration_user__del_unique_registration_user.py
migrations/0004_auto__add_registration.py
migrations/0001_initial.py
models.py

admin.py, forms.py and models.py are quite easy to fix, so lets start with that:

# models.py
.
.
.
class User(AbstractBaseUser, PermissionsMixin):
  .
  .
  .
  # Remove this line
  # receive_newsletter = models.BooleanField(_('receive newsletter'), default=False)


# admin.py
.
.
.
class UserAdmin(AuthUserAdmin):
  fieldsets = (
    # update this line
    (None, {'fields': ('username', 'password',)}), # 'receive_newsletter')}),
  .
  .
  .

  add_fieldsets = (
  # and this one
    (None, {'fields': ('username', 'password1', 'password2',)}), # 'receive_newsletter')}),
  .
  .
  .

  # also update these 3 lines
  list_display = ('username', 'email', 'short_name', 'full_name', 'is_active', 'is_staff',) # 'receive_newsletter')
  list_editable = ('is_active',) # 'receive_newsletter')
  list_filter = ('is_staff', 'is_superuser', 'is_active', 'groups',) # 'receive_newsletter')


# forms.py
.
.
.
class UserCreationForm(AuthUserCreationForm):
  # remove this line
  # receive_newsletter = forms.BooleanField(required=False)
.
.
.
class UserChangeForm(AuthUserChangeForm):
  # as well as this one
  # receive_newsletter = forms.BooleanField(required=False)

Now the migrations... This part can sound scary and it might be tricky depending on how complicated your migrations are. But most of the time it will be as simple as this: search all of them for the field to remove and delete the line mentioning it.

Once you have done this, you are ready to do your first commit:

$ hg status
? .hgignore
? __init__.py
? admin.py
? forms.py
? management/__init__.py
? management/commands/__init__.py
? management/commands/purgeregistrations.py
? migrations/0001_initial.py
? migrations/0002_auto__add_unique_user_email.py
? migrations/0003_auto__del_field_user_last_name__del_field_user_first_name__add_field_u.py
? migrations/0004_auto__add_registration.py
? migrations/0005_auto__add_field_registration_type.py
? migrations/0006_auto__chg_field_registration_user__del_unique_registration_user.py
? migrations/__init__.py
? models.py
? templates/myauth/email_sent.html
? templates/myauth/login.html
? templates/myauth/register.html
? templates/templated_email/registration.email
? tests.py
? urls.py
? views.py
$ hg add .hgignore __init__.py admin.py forms.py management migrations models.py templates tests.py urls.py views.py
adding management/__init__.py
adding management/commands/__init__.py
adding management/commands/purgeregistrations.py
adding migrations/0001_initial.py
adding migrations/0002_auto__add_unique_user_email.py
adding migrations/0003_auto__del_field_user_last_name__del_field_user_first_name__add_field_u.py
adding migrations/0004_auto__add_registration.py
adding migrations/0005_auto__add_field_registration_type.py
adding migrations/0006_auto__chg_field_registration_user__del_unique_registration_user.py
adding migrations/__init__.py
adding templates/myauth/email_sent.html
adding templates/myauth/login.html
adding templates/myauth/register.html
adding templates/templated_email/registration.email
$ hg ci -m "Re-usable custom user"

Now, since we removed any mention of the receive_newsletter field, your application is not yest suitable to be used as-is from your existing project. Let's fix that! We are going to create two branches:

  1. a newsletter_projects branch off of default (to which we will be adding back the receive_newsletter field)
  2. a babbler branch off of newsletter_projects

Here we go:

$ hg branch newsletter_projects
marked working directory as branch newsletter_projects
(branches are permanent and global, did you want a bookmark?)
$ cp ../babbler/myauth/models.py .
$ cp ../babbler/myauth/forms.py .
$ cp ../babbler/myauth/admin.py .
$ hg ci -m "adding back receive_newsletter to models, forms and admin"

Before going ahead and creating the babbler branch, we have to re-create a migration which will add a receive_newsletter field to the database. In order to do that, we will remove the old myauth app from the babbler project and replace it with the newsletter branch from our new repository using the sub-repository feature of mercurial. We will the trick South into creating a new migration.

To replace the current myauth app with the new re-usable one, run the following commands:

$ cd ../babbler
$ hg rm myauth/*
removing myauth/management/__init__.py
removing myauth/management/commands/__init__.py
removing myauth/management/commands/purgeregistrations.py
removing myauth/migrations/0001_initial.py
removing myauth/migrations/0002_auto__add_unique_user_email.py
removing myauth/migrations/0003_auto__del_field_user_last_name__del_field_user_first_name__add_field_u.py
removing myauth/migrations/0004_auto__add_registration.py
removing myauth/migrations/0005_auto__add_field_registration_type.py
removing myauth/migrations/0006_auto__chg_field_registration_user__del_unique_registration_user.py
removing myauth/migrations/__init__.py
removing myauth/templates/myauth/email_sent.html
removing myauth/templates/myauth/login.html
removing myauth/templates/myauth/lost_password.html
removing myauth/templates/myauth/lost_password_change.html
removing myauth/templates/myauth/lost_password_email_sent.html
removing myauth/templates/myauth/register.html
removing myauth/templates/templated_email/lost_password.email
removing myauth/templates/templated_email/registration.email
$ rm -rf myauth
$ hg clone path/to/myauth -r newsletter_projects
destination directory: myauth
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 25 changes to 22 files
updating to branch newsletter_projects
22 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo "myauth = path/to/myauth" > .hgsub
$ hg add .hgsub
$ hg status
M mybaseproject/urls.py
A .hgsub
R myauth/__init__.py
R myauth/admin.py
R myauth/forms.py
R myauth/management/__init__.py
R myauth/management/commands/__init__.py
R myauth/management/commands/purgeregistrations.py
R myauth/migrations/0001_initial.py
R myauth/migrations/0002_auto__add_unique_user_email.py
R myauth/migrations/0003_auto__del_field_user_last_name__del_field_user_first_name__add_field_u.py
R myauth/migrations/0004_auto__add_registration.py
R myauth/migrations/0005_auto__add_field_registration_type.py
R myauth/migrations/0006_auto__chg_field_registration_user__del_unique_registration_user.py
R myauth/migrations/__init__.py
R myauth/models.py
R myauth/templates/myauth/email_sent.html
R myauth/templates/myauth/login.html
R myauth/templates/myauth/lost_password.html
R myauth/templates/myauth/lost_password_change.html
R myauth/templates/myauth/lost_password_email_sent.html
R myauth/templates/myauth/register.html
R myauth/templates/templated_email/lost_password.email
R myauth/templates/templated_email/registration.email
R myauth/tests.py
R myauth/views.py

Now we will trick South into creating a new migration by:

  1. removing the current migration history
  2. telling it that we ran every exiting migration
  3. asking it to create a new migration to reflect the latest changes (creation of a new field)
  4. telling it again that we have ran every existing migration
$ echo "delete from south_migrationhistory;" | ./manage.py dbshell
$ ./manage.py migrate --fake
$ ./manage.py schemamigration myauth --auto
$ ./manage.py migrate --fake

Now since we have added a file to a sub-repository, mercurial won't show it to us directly if we do a hg status from the main repo.
So let's move to the myauth subdirectory and do everything from there (commiting our change and moving to a new branch):

$ cd myauth/
$ hg status
? migrations/0007_auto__add_field_user_receive_newsletter.py
$ hg add migrations/0007_auto__add_field_user_receive_newsletter.py
$ hg ci -m "created migration for receive_newsletter field"
$ hg push
pushing to ssh://hg@vc/web/myauth
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
$ hg branch babbler
marked working directory as branch babbler
(branches are permanent and global, did you want a bookmark?)

Back into the main repository we will also have to tell it we updated a subrepository to a new revision. This is almost transparent:

$ hg ci -S -m "updated myauth" $ hg push

The -S flag tells mercurial to also commit changes to sub-repositories.

Now there is one last thing we need to address: it's the following lines inside our project's urls.py:

from django.contrib import admin
from myauth.forms import AuthLoginForm

admin.site.login_form = AuthLoginForm
admin.autodiscover()

Since it is only 2 lines, it's not really usefull to move them to a separate file which would require to be imported, but since we have to make sure to remember to include those lines inside every project's urls.py when using our new app, let's add it to a README.md file at the root of our app explaining what to do.
Once this is done, let's commit and push our changes:

$ hg ci -S -m "Added readme to myauth"
$ hg push --new-branch

Note the --new-branch option. Remember that we moved from the newsletter branch to the babbler branch of our app earlier. But since there was nothing to be commited to that new branch, it's only now that mercurial is pushing the new branch name to the main repository

There you go, now you are all set to keep using your main project as before and also being able to reuse the code-base you created for your custom user model. All you have to do in order to use that app inside another project is:

$ hg clone path/to/myauth
$ cd myauth
$ hg branch my_project_name
$ cd ..
$ echo "myauth = path/to/myauth" > .hgsub
$ hg add .hgsub
$ hg ci -m "added myauth to the project"
$ hg push

And don't forget to follow the instructions inside your README Wink

Any changes you now make to your app should be either:

  • commited to the default branch if it is a change applying to every project (ie: bugfix). The various branches of the project should then merge the default branch
  • commited to the project specific branch (the default behaviour)

To import changes made to myauth's default branch in a project using the app, pull and merge default (then commit your changes from the main project directory)

$ cd reusable_app_path
$ hg pull
$ hg merge default
$ cd ..
$ hg ci -S -m "updated reusable_app"
$ hg push

Well, that's it for now, we will be back soon with the second part of this tutorial: creating re-usable re-distributable apps. In the mean time: Happy coding everyone!

Tutorial South Django

Comments

Post a comment