More Advanced Apps¶
This documentation carries on the Quickstart.
Apps with dependencies¶
When you wrote your first app (
jobs.myapp.MyFirstApp) you had to set
app_name on the class. That’s how you reference other apps when setting
up dependency management. This is important to note. The name of the Python
file or the name of class does not matter.
Diving in, let’s now create two more apps. For simplicity we can continue
in the file
myapp.py you created. Add this:
class MySecondApp(BaseCronApp): app_name = 'my-second-app' depends_on = ('my-first-app',) # NOTE! def run(self): with open(self.app_name + '.log', 'a') as f: f.write('Second app run at %s\n' % datetime.datetime.now())
Exactly where you add this app doesn’t really matter. Before or after or even
in a different file. All the matters is the
app_name attribute and
depends_on. It event doesn’t matter which order you place it in
jobs setting. You can change your
crontabber.ini to be like this:
jobs=''' jobs.myapp.MySecondApp|1h jobs.myapp.MyFirstApp|5m '''
crontabber reads the
jobs setting but when there’s dependency linking
apps, even though it reads the
jobs setting from top to bottom, it knows
jobs.myapp.MySecondApp needs to be run after
Go ahead and try it:
If you now look at the timestamps in
are in the correct order according to the dependency linkage rather than the
order they are written in the
Another important thing to appreciate is that if a job fails for some reason,
i.e. a python exception is raised, it will stop any of the dependening jobs
from running. Basically, if job
B depends on job
B will not
run until job
A ran without a failure. Basically,
crontabber not only
makes sure the order is correct, it also guards from running dependents if
their “parent” fails.
About the job frequency¶
In the above example note the notation used for the
The frequency is pretty self explanatory.
5m means every 5 minutes
1h means every hour. The other thing you could use is, for example,
3d meaning every 3 days.
Running at specific times¶
Suppose you have a job that is really intensive and causing a lot of stress in your server. Then you might want to run that “at night” (in quotes because it means different things in different parts of the world) or whenever you have the least load in your server.
The way to specify time is to write it in
HH:MM notation on a 24-hour
The way you specify the time is to add it to the
jobs second like this
jobs=''' jobs.myapp.MyBigFatWeddingApp|2d|21:00 ```
But here’s a very important thing to remember. The timezone is that of your
PostgreSQL server. Not the timezone of your server.
However, when you install PostgreSQL it will take the timezone from your
server’s timezone. So if you have a server on the US west coast, the default
timezone will be
However, you can, and it’s a good idea to do, change the timezone of your
PostgreSQL server. So if you have set your PostgreSQL server to
crontabber will adjust these times in
Postgres specific apps¶
crontabber provides several class decorators to make use of postgres
easier within a crontabber app. These decorators can imbue your app class
with the correct configuration to automatically connect with Postgres and
handle transactions automatically. The three decorators provide differing
levels of automation so that you can choose how much control you want.
This decorator tells crontabber that you want to use postgres by adding to
your class two class attributes:
self.database_transaction_executor. When execution reaches your run
method, you may use these two attributes to talk to postgres. If you want
a connection to Postgres you can grab one from the
database_connection_factory and use it as a context manager:
# ... with self.database_connection_factory() as pg_connection: cursor = pg_connection.cursor()
The connection that you get from the factory is a psycopg2 connection, so you have all the resources of that module available for use with your connection. You don’t have to worry about opening or closing the connection, the context manager will do that for you. The connection is open and ready to use when it is handed to you, and is closed when the context ends. You are responsible for transactions within the lifetime of the context.
If you want help with transactions, there is also a the
database_transaction_executor at your service. Give it a function that
accepts a database connection as its first argument, and it will execute the
function within a postgres transaction. If your function ends normally (with
or without a return value), the transaction will be automatically committed.
If an exception is raised and that exception escapes outside of your function,
then the transaction will be automatically rolled back.
@using_postgres() class MyPGApp(BaseCronApp): def execute_lots_of_sql(connection, sql_in_a_list): '''run multiple sql statements in a single transaction''' cursor = connection.cursor() for an_sql_statement in sql_in_a_list: cursor.execute(an_sql_statement) def run(self): sql = [ 'insert into A (a, b, c) values (2, 3, 4)”, 'update A set a=26 where b > 11', 'drop table B' ] self.database_transaction_executor( execute_lots_of_sql, sql_in_a_list )
This decorator is to be used in conjunction with the previous decorator. When using this decorator, your run method must be declared with a database connection as its first argument:
@using_postgres() @with_postgres_connection_as_argument() class MyCrontabberApp(BaseCronApp): app_name = 'postgres-enabled-app' def run(self, connection): # the connection is live and ready to use cursor = connection.cursor() # ...
With this decorator, the database connection is handed to you. You don’t have to get it yourself. You don’t have to worry about closing the connection, it will be closed for you when your ‘run’ function ends. However, you are still responsible for your own transactions: you must explicitly use ‘commit’ or ‘rollback’. If you do not ‘commit’ your changes, they will be lost when the connection gets closed at the end of your function.
You still have the transaction manager available if you want to use it. Note, however, that it will acquire its own database connection and not use the one that was passed into your run function. Don’t deadlock yourself.
This decorator gives you the most automation. It considers your entire run function to be a single postgres transaction. You’re handed a connection through the parameters to your run function. You use that connection to accomplish database stuff. If your run function exits normally, the ‘commit’ will happen automatically. If your run function exits with a Exception being raised, the connection will be rolled back automatically.
@using_postgres() @as_single_postgres_transaction() class MyCrontabberApp(BaseCronApp): app_name = 'postgres-enabled-app' def run(self, connection): # the connection is live and ready to use cursor = connection.cursor() cusor.execute('insert into A (a, b, c) values (11, 22, 33)') if bad_situation_detected(): raise GetMeOutOfHereError()
In this example, connections are as automatic as we can make them. If the exception is raised, the insert will be rolled back. If the exception is not raised and the ‘run’ function exits normally, the insert will be committed.
crontabber is all Python but some of the tasks might be something other
than Python. For example, you might want to run
or something more advanced.
What you do then is use the
When you use this helper on your application class, you can use
self.run_process() and it will return a tuple of exit code, stdout, stderr.
This example shows how to use it:
from crontabber.base import BaseCronApp from crontabber.mixins import with_subprocess @with_subprocess class MyFirstCommandlineApp(BaseCronApp): app_name = 'my-first-commandline-app' def run(self): command = 'rm -f /var/logs/oldjunk.log' exit_code, stdout, stderr = self.run_process(command) if exit_code != 0: self.config.logger.error( 'Failed to execute %r' % command, ) raise Exception(stderr)