Slides from my presentation at the “Improving Scientific Software” conference at the National Center for Atmospheric Research (NCAR).
Using RQ for scheduling tasks
RQ and remote scheduling
Reader level: Introductory RQ can be used to set up queues for executing long-running tasks on local or remote machines. Some steps on how to install and get started with RQ are listed below.
Installation Create a virtual environment and we will have to install the following components:
Redis-server RQ RQ-scheduler Install Redis using the following
wget http://download.redis.io/redis-stable.tar.gz tar xvzf redis-stable.tar.gz cd redis-stable make Run ‘make test’ to make sure things are working properly, followed by ‘sudo make install’ to complete the installation.
[Read More]
Virtual environments for Anaconda Python
Useful conda commands
Installation using Conda To create a conda environment named ‘myenv’:
conda create --name myenv To create an environment from a file ‘test.yml’:
conda env create -f test.yml The environment name comes from the line ‘name: tag’ inside the ‘test.yml’ file.
To create a named environment from a file ‘test.yml’:
conda env create -f test.yml -n pytorch To create an environment from the base environment:
conda create --name myenv --clone base To remove an environment named ‘envname’:
[Read More]
Data Scientist’s Python Toolbox - Introduction to Pandas
Workshop slides
This page contains slides to the ‘Data Scientist’s Python Toolbox’ workshop that I have taught at the PEARC conference.
Introduction to Python for Scientific Computing
Workshop slides
This page contains slides to the ‘Python for Scientific Computing’ workshop that I have taught at Virginia Tech and at the XSEDEconference.