resources

2015-08-03 14:02 | Source

A - totally incomplete - list of resources I have come across on Python and Python for data analysis and visualization, loosely organized by category:

Object Oriented Programming in Python

A good tutorial by Alan Gauld
The classes entry from the official Python doc
A short pdf from Andrew Walker's Python for Earth Scientists course

Python as a glue: wrapping of C, C++, Fortran or the Cython module

Real-time acquisition, physical programming

pyserial: Module that encapsulates the access for the serial port
Real World Instrumentation, Automated Data Acquisition and Control Systems: A book available on Amazon.
Programming Raspberry Pi: Getting Started with Python: book (not free) with code (free)
PyVISA: The PyVISA package enables you to control all kinds of measurement equipment through various busses (GPIB, RS232, USB) with Python programs

Image processing

PIL (Python Image Library): Image processing capabilities to your Python interpreter.
pillow: A more user-friendly fork of PIL
scikit-image: more Image processing, built on top of Numpy / Scipy.

Symbolic maths

The SymPy library is a Python library for symbolic mathematics. It supports polynomials, calculus, solving equations, etc
The sage software: Mission: Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab

Geospatial statistics

PySAL: PySAL is a cross-platform library of spatial analysis functions written in Python. It is intended to support the development of high level applications for spatial analysis.
GeoPandas: A project based on Pandas to make working with geospatial data in python easier
Rasterio: Clean and fast and geospatial raster I/O with Numpy support, developed by the team at https://www.mapbox.com/
Pyproj: Performs cartographic transformations and geodetic computations. Wrapper around the Proj version 4 library
Python GDAL/OGR: Python bindings + tools around the Geospatial Data Abstraction Library
Python GIS resources: a blog on geospatial python
High Performance Geostatistics Library: A library written in C++ / Python implementing geostatistical algorithms (e.g. kriging, correlograms, etc)

Biology, ecology

Biopython: Biopython is a set of freely available tools for biological computation written in Python by an international team of developers.
GeoEco: Open source geoprocessing toolbox designed for coastal and marine researchers and GIS analysts who work with spatially-explicit ecological and oceanographic data. For Windows (> XP) only.
Galaxy: Galaxy is a scientific workflow, data integration and data and analysis persistence and publishing platform that aims to make computational biology accessible to research scientists that do not have computer programming experience.

Computational Fluid Dynamics and PDE solvers

Python on the GPU

PyCUDA: PyCUDA lets you access Nvidia‘s CUDA parallel computation API from Python

Parallel computing with Python / IPython

IPython parallel introduction
Interactive Parallel computing with IPython: tutorials by Min. Raglan-Kelley at Scipy2014 (3 parts in the youtube list)
IPython parallel for distributed computing: A talk at PyCON Australia by Nathan Faggian
IPython & Jupyter in depth: high productivity interactive and parallel python: A talk at PyCON 2015 by Thomas Kluyver and Kyle Kelley

Signal processing in Python

Python for Matlab and R users

Some URLs and blogs

Python Scientific Lectures Notes: Tutorial material on the scientific Python ecosystem, a quick introduction to central tools and techniques.
Python for earth scientists: A two afternoons course by Andrew Walker (University of Bristol) on Python in the earth sciences.
oceanpython: Python for oceanography
PyAOS: Python for the Atmospheric and Oceanic Sciences
python4oceanographers: Learn python with examples applied to marine sciences.
Pythonic perambulations: A blog by Jake VanderPlas

Github repositories: notebooks and accompanying material

A gallery of interesting IPython Notebooks "...a curated collection of IPython notebooks that are notable for some reason."
Scikit-learn tutorial: Files and other info associated with the Scipy 2013 scikit-learn tutorial developed by Gaël Varoquaux, Olivier Grisel and Jake VanderPlas.
Statistical Analysis tutorial from Chris. Fonnesbeck.
Bayesian Statistical Analysis in Python: Ipython notebooks for the Scipy 2014 tutorial on Bayesian data analysis with Python, by Chris. Fonnesbeck.
AstroML: Machine Learning and Data Mining for Astronomy: A library and tutorial by Jake VanderPlas and co-authors, accompanying the book Statistics, Data Mining, and Machine Learning in Astronomy
Scientific Python lectures from Robert Johansson
Data Science in Python: A series of annotated notebooks on data science (i.e. geared towards machine learning) in python
http://earthpy.org/: EarthPy is a collection of IPython notebooks with examples of Earth Science related Python code
data science notebooks: Updated Data Science Python Notebooks: Spark, Hadoop MapReduce, HDFS, AWS, Kaggle, scikit-learn, matplotlib, pandas, NumPy, SciPy ...
Effective computation in physics: the code examples accompanying the book

Some books

Python for Data Analysis: From Wes McKinney (Developer of Pandas)
Think stats: Probability and statistics for programmers, from Allen Downey, pdf available for free.
Think complexity: Complexity science (graphs, cellular automata, agent-based models), from Allen Downey, pdf available for free.
Python in hydrology: A book freely available in pdf, from Sat Kumar Tomer.
Programming collective intelligence: By Toby Segaran, Good intro on (general) Machine Learning algorithms.
Machine Learning in action: By Peter Harrington.
Introduction to Python for Econometrics, Statistics and Numerical Analysis: Second Edition: By Kevin Sheppard, Oxford Uni.
A Hands-On Introduction to Using Python in the Atmospheric and Oceanic Sciences: By Johnny Lin, Professor of physics and head of the Climate Research Group at North Park University.
principles of planetary climate: by Ray Pierrehumbert. Excellent book on the physics of planetary climates, with freely downloadable python code to follow the examples given in the book.
Effective computation in physics: by Katy Huff and Anthony Scopatz, Field Guide to Research with Python. Excellent. Code examples at https://github.com/physics-codes.

Some interesting libraries, built on top of the main Scientific stack

xray: N-D labeled arrays and datasets in Python.
PYMC: By Chris Fonnesbeck, Bayesian statistical models and fitting algorithms, including Markov chain Monte Carlo.
Seaborn: Statistical data visualization, by Michael Waskom. Its graphical representation of linear models is particularly interesting.
ggplot: For R users, a 'port' of the ggplot2 package to Python, see here for what's new in the latest release.
coards: A COARDS compliant time parser. See also netcdftime which is part of the NetCDF4 module
seawater: Similar to the MATLAB toolboxes SEAWATER from CSIRO and parts of OCEANS from Woods Hole Institute.
fluid: Procedures to study fluids on Python, focused for oceanography, meteorology and related sciences.
kyPyWavelet: Continuous wavelet transform module for Python ala Torrence and Compo. Some manual edits were necessary to make it work for me ...
pyresample: Resampling (reprojection) of geospatial image data in Python
Rpy2: calling R from Python

Some articles on open and reproducible research

Reproducible Research in Computational Science, Roger D. Peng, Science 334, 1226 (2011).
Shining Light into Black Boxes, A. Morin et al., Science 336, 159-160 (2012).
The case for open computer programs, D.C. Ince, Nature 482, 485 (2012).
Best practices for scientific computing: Paper in PLOS Biology exposing some of the tools and methods to build better Scientific software.