4 posts tagged with "python"

So, you've got lots of Python code being executed from your Elixir app, and lots of ad-hoc calling of your Python code. On one hand, you want to dynamically scale to manage demand, but at the same time, you don't want to accidentally crash your application by starting too many Python processes. So... what what's a frazzled developer gotta do to save himself some headache? The key problem that we are trying to solve here pooling, which is the the management of limited resources kept at the ready for any ad-hoc requests for usage.

As explained in my original post on connecting Python with Elixir, there are some data translation issues that we will have to deal with when interfacing between the two languages.

Thankfully, the Erlport documentation kindly provides a full table of the data type mappings, which we can now use to make our lives easier.

While developing PillowSkin, I have become used to the habit of creating small releases. This resulted in me always having to repetitively write the commands git checkout master, then git merge develop, then git tag... It does get old after a while when your total release count gets into the hundreds (PillowSkin is at ~180 versions at this point in writing). However, with the power of Python scripting, we can do a little bit of scripting to save us countless cumulative hours of googling for the right commands to type. This post details the creation of a simple CLI tool to automate release building using Invoke, a Python task runner, which I explain in my main post here

Pre-requisites:

• Git must be installed, of course.
• Python 3, preferably. Invoke works with Python 2, but it has since reached EOL, so always write in Python 3 if you can.

GitFlow Basics​

In a nutshell, GitFlow is all about two main branches: a master branch and a develop branch. All new features are written in their own branches and are merged into the develop branch. Once a release is due, the release is created from the develop branch and then the develop branch is merged into the master branch.

This ensures that the master branch is always representative of your production source code, and is always at a specific version. You can read about the full GitFlow strategy here, I highly recommend adopting a simplified version of it for your development workflow.

Create Your tasks.py File​

If you haven't already done so, create your tasks.py file in your working directory.

Add in the following:

from invoke import task

@task
def tag(c):
    """
    Tags a new git release, automates merging develop into master.
    Follows semantic versioning.
    """
    print("it works!")
    pass

This is our placeholder function for our new script. This function will be available from the command line with the following command:

$ inv tag
it works!

Extracting Our Current Version​

Because humans only have a limited working memory, there isn't any point in trying to memorize what version your project is currently at. Let's write some code to check it for us. Remove the placeholder pass and print, and type the following:

def tag(c):
    current_ver = c.run('git describe', hide=True, warn=True).stdout

This line runs the command git describe, hiding anything that is printed out to stdout (so that we can nicely format the output), and then take the received stdout property of the returned result of the run function.

New Version Input​

We now need to get the new desired version to bump up to. Add in the following line:

def tag(c):
    current_ver = c.run('git describe', hide=True, warn=True).stdout
    version = input('[INPUT] What version is this release? e.g. 0.1.0\n'
                    'Current version is: {}\n\n'.format(current_ver))

I like to use square brackets to indicate what type of action is being taken by the script, but the [INPUT] is completely optional. This line asks the user to type in new version number. Note that an example should be given to increase usability (and let the user think less about whether he/she should include a v).

Git Tag Message​

Now, git requires us include a tag message, to serve as a description of the tag. So, just as we did before, we'll include another input function to ask for user input.

def tag(c):
    current_ver = c.run('git describe', hide=True, warn=True).stdout
    version = input('[INPUT] What version is this release? e.g. 0.1.0\n'
                    'Current version is: {}\n\n'.format(current_ver))
    msg = input('[INPUT] What is this release about? e.g. Some cool message\n')

Git Branch Merging and Tagging​

Now that we have all the necessary information, we'll do the automation for the branch merging.

We'll be doing it sequentially:

1. Merge develop into master
2. Tag the master branch
3. Push everything to our remote repository (origin)
4. Checkout to develop for continued development

def tag(c):
    current_ver = c.run('git describe', hide=True, warn=True).stdout
    version = input('[INPUT] What version is this release? e.g. 0.1.0\n'
                    'Current version is: {}\n\n'.format(current_ver))
    msg = input('[INPUT] What is this release about? e.g. Some cool message\n')
    # Merge develop into master
    print('[ACTION] Merging develop branch into master.')
    c.run('git checkout master', hide=True)
    c.run('git merge develop', hide=True)
    print('[RESULT] Merged develop branch into master')
    
    # Tag the master branch
    print('[ACTION]  Tagging project for v{} with message "{}"'.format(version, msg))
    c.run('git tag -a v{} -m "{}"'.format(version, msg), hide=True)
    updated_ver = c.run('git describe', hide=True).stdout
    print('[RESULT] Updated current version is: {}'.format(updated_ver))

    # Keep main branches on origin updated
    print('[ACTION] Pushing to origin')
    c.run('git checkout master', hide=True)
    c.run('git push origin master', hide=True)
    c.run('git checkout develop', hide=True)
    c.run('git push origin develop', hide=True)
    # Push tags
    c.run('git push origin --tags', hide=True)
    print('[RESULT] Completed push to origin')

    c.run('git checkout develop')

This is the entire script required. Some things to note:

• Tagging automatically adds in the v before the version numbers. By convention, many projects include the v for their versioning, for example v0.X.X, which I too think is quite clear that it is a version string.
• We print out the new updated tag version for the user, so that the user knows that the desired action is complete.

This script should save you quite a few minutes when managing your releases, and also encourages small releases. What's there not to like?

Adding Python into your Elixir web application is a very tantalizing proposition. Elixir isn't that great at crunching numbers, but by leveraging the data science powers of Python, we can add in some machine learning magic into our applications.