ulmo¶
Ulmo is a python library for clean, simple and fast access to public hydrology and climatology data.
Installation¶
Ulmo depends on a lot of libraries from the scientific python stack (namely: numpy, pytables and pandas) and lxml. There are a couple of ways to get these dependencies installed but it can be tricky if doing it by hand. The simplest way to get things up and running is to use a scientific python distribution that will install everything together. A full list is available on the scipy website but Anaconda / Miniconda is recommended as it is the easiest to set up.
If you are using Anaconda/Miniconda then you can install ulmo from the IOOS channel with the following command:
conda install -c ioos ulmo
Otherwise, follow the instructions below:
Once the requisite scientific python libraries are installed are installed, the most recent release of ulmo can be installed from pypi. Pip is a good way to do that:
pip install ulmo
To install the bleeding edge development version, grab a copy of the source code and run setup.py from the root directory:
python setup.py install
To setup a development environment using conda:
conda env create -n myenv –file py2_conda_environment.yml (or py3_conda_environment.yml if you want to work with python 3) source activate myenv (use ‘activate test_environment’ on windows) python setup.py develop
- optionally run tests:
- python setup.py test
Quick Start¶
Check out this brief but awesome tutorial by Nikolay Koldunov to get a sense of what Ulmo is about.
Getting help¶
If your stuck, have questions, ideas, or just want to say hi feel free to drop us a line on the mailing list.
Contributing¶
If you’d like to contribute code or documentation, visit our github repository. Bug reports, issues and especially pull Requests are welcome.
API Reference¶
- California Department of Water Resources Historical Data
- Climate Prediction Center Weekly Drought
- CUAHSI WaterOneFlow
- National Climatic Data Center Climate Index Reference Sequential (CIRS)
- National Climatic Data Center Global Historical Climate Network Daily
- National Climatic Data Center Global Summary of the Day
- Texas Weather Connection Daily Keetch-Byram Drought Index (KBDI)
- US Army Corps of Engineers - Tulsa District Water Control
- USGS National Water Information System
- USGS Emergency Data Distribution Network services
- USGS Earth Resources Observation Systems (EROS) services
- USGS National Elevation Dataset (NED) services
Changelog¶
0.8.0 (released 2015-09-15)
- python 3 compatibility. ulmo is now compatible py python 2.7 and python 3.4.
- ulmo is now being tested on Windows/Linux/MacOSX using AppVeyor and Travis CI.
- the ioos conda channel is now the supported way to get ulmo.
- added conda environment yaml files to make it easier to set up a development environment.
0.7.8 (released 2015-07-02)
- no changes, had to bump version number because of a corrupt file uploaded to pypi.
0.7.7 (released 2015-07-02)
- dropped python 2.6 support
- fixed bug in extracting raster file from zip file on windows
- fixes for cdec service
- fix url_params kwarg typo for nwis service
0.7.6 (released 2015-04-27)
- fixed ncdc.gsod services to use new station list over depreciated list (thanks Victor)
- added USGS National Elevation Dataset (NED) raster service
- added USGS Earth Resources Observation Systems (EROS) raster services
- allow passthrough of extra kwargs for USGS NWIS services
0.7.5 (released 2015-01-26)
- add support for pandas 0.14 and 0.15
- add a function to selectively remove data from nwis hdf5 cache
0.7.4 (released 2014-09-09)
- in cuahsi.wof module: allow optional configuration of caching behavior for underlying suds SOAP library
- in cuahsi.wof.get_values(), variable_code is no longer a keyword argument. It didn’t make sense for it to be a kwarg since it is not optional.
0.7.3 (released 2014-07-16)
- bugfix: using parameters argument in ulmo.ncdc.gsod.get_data() was not working
0.7.2 (released 2014-06-17)
- add parser for TWDB DOT dataloggers to usgs.eddn
0.7.1 (released 2014-06-05)
- bugfix: in cpc.drought dataset calculate year number externally rather than using year number from data files, as they are sometimes incorrect (see github issue #66)
0.7.0 (released 2014-04-01)
this release includes some backwards incompatible changes
update ncdc.cirs to use new nClimDiv dataset (see ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/div-dataset-transition-readme.txt) for details about the new dataset
- some ncdc.cirs element names have changed:
- cdd -> cddc
- hdd -> hddc
- pcp -> pcpn
- tmp -> tmpc
fix flakey cpc.drought url determination causing data for the 2011 agricultural year to be unavailable
0.6.7 (released 2014-03-28)
- fix usgs.cpc year/week number reckoning bug
0.6.6 (released 2014-03-24)
- fix usgs.eddn parsers: values should be interpretted as reversed for sutron and texuni dataloggers
0.6.5 (released 2014-03-14)
- usgs.eddn hdf5 files will no longer grow larger than they have to with each update
- fix incorrect ghcn_daily wm_oid when using pandas 0.13.1
- fix incorrect pytables required version
0.6.4 (released 2014-01-22)
- a bunch of minor fixes for working with pandas 0.13
- this release drops support for pandas 0.10.1; the minimum required version of pandas is now 0.11
- usgs.eddn.get_data will now make multiple requests if it encounters a ‘Maximum data limit reached’ message
0.6.3 (released 2013-12-12)
- add cdec.historical for accessing California Department of Water Resources California Data Exchange Center
- fix last_modified dates being incorrectly updated when using nwis hdf5 backend on some platforms
0.6.2 (released 2013-11-19)
- add usgs.eddn module for accessing USGS Emergency Data Distribution Network
0.6.1 (released 2013-07-31)
- raise a friendlier error message for usace.swtwc when no data is found
- bugfix: waterml files with empty metadata elements (e.g. method) were breaking the parsing logic
0.6.0 (released 2013-07-30)
- this release includes some backwards incompatible changes
- simplified labels used for KBDI values: kbdi_avg -> avg, kbdi_min -> min...
- added parameter_code kwarg to usgs.nwis.hdf5.get_site_data
- added start and end dates to cuahsi.wof.get_values()
- added support for NCDC’s CIRS drought index dataset
- fixed filesystem timestamp related bug that could cause file-based datesets to be unnecessarily re-downloaded
- version number is now available at ulmo.__version__
- misc documentation fixes
0.5.0 (released 2013-05-14)
- nwis.pytables has been replaced with nwis.hdf5 which uses the pandas.io.pytables backend; things should generally run a bit faster now and the code is much cleaner. There is one backwards-incompatible change: nwis.hdf5.get_site() is now thread-safe. There was an undocumented behavior where if a site could not be found in the hdf5 file during a nwis.pytables.get_site() call, then a request to the NWIS service would be made and the site table would be updated but no longer happens. You must explicitly run nwis.hdf5.update_site_list() to update the site list first.
- the nwis.pytables namespace is deprecated in favor of nwis.hdf5 because it is shorter and more recognizable but pytables is still powering things under the hood (via the pandas HDFStore)
- parsed timeseries data dicts from waterml-based datasets (cuahsi.wof and usgs.nwis) now contain parsed site info mapped to the ‘site’ key
- add support for pre-1997 data from CPC’s weekly drought monitor dataset
- bugfix: some related to change of the “current” agricultural year for CPC weekly drought data
- add support for Texas Weather Connection’s Daily Keetch-Byram Drought Index (KBDI) dataset
0.4.1 (released 2013-03-04)
- bugfix: interpret missing data (‘—-‘) correctly in usace.swtwc.get_stations
0.4.0 (released 2013-03-04)
- this release includes some backwards incompatible changes
- usgs.nwis date_range parameter has been replaced with more explicit start, end and period parameters
- ncdc.gsod.get_data() parameters renamed: start_date -> start and end_date -> end to be consistent with other dataset APIs
- ncdc.gsod.get_stations() now allows limiting stations to matching parameters (e.g. country=’US’)
- all date and datetime parameters are now consistently parsed
- added support for CUAHSI’s HIS Central service: ulmo.cuahsi.his_central
- added preliminary support for CPC’s weekly drought monitor dataset (>=1997)
- added preliminary support for data from USACE Tulsa Water District Control
0.3.0 (released 2013-01-14)
- this release includes some backwards incompatible changes
- moved module for USGS NWIS from ulmo.usgs -> ulmo.nwis.usgs
- moved module for WaterOneFlow from ulmo.wof -> ulmo.cuahsi.wof
0.2.4 (released 2013-01-14)
- core API functions can be called from main dataset modules
- support python 2.6
0.2.3 (released 2013-01-13)
- add ghcn daily dataset
- add preliminary support for CUAHSI WOF
- add sphinx documentation
- clean up tests
- make pytables an optional dependency; pytables-related functionality is just disabled if pytables can’t be found
0.2.2 (released 2012-11-05)
- bugfix: broken usgs.pytables last_refresh handling
0.2.1 (released 2012-11-04)
- turn on compression for pytables cache files
- minor bugfix and test cleanup
0.2.0 (released 2012-10-12)
- initial public release as ulmo (formerly pyhis)