Marine Geospatial Ecology Tools
Getting Started with Marine Geospatial Ecology Tools
| MGET version: | 0.5 |
| Document version: | $Id: GettingStarted.xsl 241 2008-06-01 16:41:38Z jjr8 $ |
| Document status: | Complete |
| Maintainer email: | jason.roberts@duke.edu |
| MGET home page: | http://code.env.duke.edu/projects/mget |
Marine Geospatial Ecology Tools (MGET) is an open source programming library designed for coastal and marine researchers and GIS analysts who work with spatially-explicit ecological and oceanographic data in scientific or managerial workflows. The initial MGET releases focus on tools useful in habitat modeling, including tools for processing and sampling remotely-sensed oceanographic data and for creating and evaluating statistical models. Also included are many powerful general-purpose data processing tools, such as batch-processing versions of popular ArcGIS Spatial Analyst tools. Subsequent MGET releases will include more advanced tools presently under development, such as tools for hydrodynamic modeling, connectivity modeling, and spatially-explicit fishery modeling.
MGET was designed with an emphasis on maximizing reliability and usability. We are all too familar with the abundance of undocumented, confusing, and difficult-to-install software that works great on the original developer's machine but not your own. We try to minimize these headaches by providing the same features you expect from the highest quality software, including an installer, documentation, extensive error-handling code, and adequate testing before we release new versions. We are professional software engineers who take pride in our work and hope that if you find a bug or see something that could be improved, please report it to Jason Roberts (jason.roberts@duke.edu). Thanks for your assistance!
You may notice that the Reference Documentation and the MGET source code use the word GeoEco rather than the phrase Marine Geospatial Ecology Tools. The two are synonymous.
If you have done some computer programming, you may be familiar with the need to invent words for use in source code. These words are typically short and allow programmers to quickly distinguish one library of functions from another. We selected GeoEco as the code-word for Marine Geospatial Ecology Tools because it is catchy, it shows up infrequently in Internet search engines relative to other choices such as MGET, and it does not suggest "marine". This last point may seem contrary to our mission, but want to offer our tools to terrestrial users, even if we currently market the tools mainly to marine users. Many of the existing tools are equally applicable above and below the waterline.
Minimum requirements:
Additional requirements for full functionality:
Much of MGET is written to be operating-system independent. We will enable installation on other operating systems in a future release.
Although you must be logged on as an administrator to install MGET, you do not need to be an administrator to run the tools once they are installed.
Exit all instances of ArcCatalog, ArcMap, ArcGlobe, etc.
You can skip this step if you do not have ArcGIS and do not want to install it.
The MGET setup program uses the .NET Framework to register the Marine Geospatial Ecology Tools ArcGIS toolbox with ArcCatalog. If the proper version of the .NET Framework is not already installed when you run the MGET setup program, you will receive an error.
ArcGIS 9.1 users:
ArcGIS 9.1 requires the .NET Framework 1.1. Windows Server 2003 users can skip this step because Server 2003 includes the .NET Framework 1.1.
ArcGIS 9.2 users:
ArcGIS 9.2 requires the .NET Framework 2.0 or later. Windows Vista users can skip this step because Vista includes the .NET Framework 3.0.
You can skip this step if you do not have ArcGIS and do not want to install it.
If ArcGIS is not already installed, install it now. Because you installed the .NET Framework first, the ArcGIS setup program will automatically install ArcGIS NET Support unless you disable it in the setup options. Do not disable it.
If ArcGIS is already installed, you must verify that .NET Support is installed, and install it if it is missing:
You can skip this step if you do not have ArcGIS and do not want to install it.
There is no harm in applying the same ArcGIS service pack more than once.
MGET is implemented in the Python programming language as a package called GeoEco. Before you installing it, you must install the proper versions of Python and the pywin32 package. But first, there are a few things you need to know about Python.
Python version numbers
Each release of the Python language interpreter is assigned a version number of the form X.Y, where X is the major number and Y is the minor number. The major number is incremented when there is a fundamental change to the language. The minor number is reset to zero whenever the major number changes, and is incremented when new features are added to the major release.
Occasionally, the Python development team will fix a bunch of bugs and issue a "bug fix" release. These releases have the form X.Y.Z where Z is the bug fix release number, starting with 1. For example, 2.5.1 is the first bug fix to Python 2.5.
Multiple releases of Python can be installed on a Windows computer at the same time. For example, you can have Python 2.1 and 2.5 installed simultaneously. But whenever you install a bug fix release, it overwrites the installation having the same X.Y number. For example, If you have Python 2.5 installed and you install 2.5.1, it will overwrite your 2.5 installation and now you will only have 2.5.1 on your machine. If you then install 2.5.2, you will only have 2.5.2 on your machine.
Unless you override the installation options in the Python setup program, Python releases are installed to C:\PythonXY. For example, if you install Python 2.1 and 2.5, they will be installed to C:\Python21 and C:\Python25. When you install bug fix releases, they overwrite the files in these directories with updated versions. We recommend you use the default installation directory C:\PythonXY unless you are an expert user.
Windows file association determines which Python version is used to execute scripts
When you have multiple versions of Python installed, Windows must decide which version to use when you double-click a Python (.py) script to run it from Windows Explorer. Windows maintains a "file association" database that associates programs to file types based on their file extensions. In general, whenever you install new software, the setup program modifies the file associations to tell Windows to use the new program to open the types of files it works with. Whenever you install a version of Python, it reconfigures the file associations so that version of Python will be used to execute scripts. For example, if you have Python 2.1 and then install 2.4, from that point forward 2.4 will be used to run Python scripts. If you then installed 2.3, it would be used even though 2.4 is installed.
You can change the file associations manually; see Microsoft Knowledge Base article 307859. To change from Python 2.3 to 2.4, for example, you would change the associations for .py, .pyc and .pyo files from C:\Python23\python.exe to C:\Python24\python.exe and .pyw from C:\Python23\pythonw.exe to C:\Python24\pythonw.exe. If you are uncomfortable doing this, you can always reinstall Python 2.4 to restore the file associations.
The main things to remember are:
ArcGIS relies on Windows file associations
ArcGIS uses the Windows Command Processor (cmd.exe) to execute Python scripts. cmd.exe uses Windows file associations to determine which program should be used to execute .py files.
Which version of Python should I use if I have ArcGIS?
ArcGIS 9.1 users:
ArcGIS 9.1 shipped with Python 2.1. The MGET Python package, GeoEco, requires Python 2.4 or later. If you do not have a preference, we recommend the latest version. As of this writing, we have used 2.4 with Arc 9.1 for two years and 2.5 for one year and are confident that both are compatible.
Important: Do not uninstall your existing version of Python 2.1! Even though ArcGIS relies on Windows file associations to select the version of Python for script execution, it explicitly checks for the presence of Python 2.1. It also checks for the "Python 2.1 combined Win32 extensions" (the win32all package). If you accidentally uninstall these, you can download them from here: Python 2.1.3, win32all. After reinstalling them, you will must restore your file associations to your later version of Python by reinstalling it or manually editing the associations.
ArcGIS 9.2 users:
According to ESRI Technical Article 31912, ArcGIS 9.2 is "hard wired" to work with Python 2.4.1, and using any other version is not supported. Based on the strong wording of ERSI's article, we recommend you stick with Python 2.4.1 unless you are an expert programmer.
In our experiments, 9.2 will work with Python 2.5 but only if the Python script instantiates
the geoprocessor object using the old technique, invoking win32com.client.Dispatch.
The new technique, invoking arcgisscripting.create(), will not work. This is
is because the arcgisscripting module is implemented as a Python "extension DLL"
and therefore can only work with a specific version of Python. If you examine the
DLLs required by C:\Program Files\ArcGIS\Bin\arcgisscripting.dll, you will see
python24.dll. If you try to import arcgisscripting from Python 2.5, it will raise
ImportError: No module named arcgisscripting.
GeoEco is written to try win32com.client.Dispatch first. If it
fails, it tries arcgisscripting.create(). Under this logic, GeoEco
can successfully instantiate the geoprocessor using with Python 2.4 or any later
GeoEco-supported version so long as the pywin32 Python package is installed, but
Python 2.4 is still supported if pywin32 is not installed. Eventually GeoEco
will be changed to try the arcgisscripting.create() method first,
but this will likely only happen after ESRI fixes the ArcGIS bug described in
ticket #152.
We regularly run GeoEco with Python 2.5 and ArcGIS 9.2 without problems.
If you are an expert programmer you can do the same. But beware: ArcGIS 9.2
includes some geoprocessing tools implemented as Python scripts. These scripts
all use the import arcgisscripting approach and will therefore
fail under Python 2.5.
ArcGIS 9.2 says it requires 2.4.1. Can I install a newer bug fix release, such as 2.4.4?
We have had no problems installing newer bug fix releases, and we recommend it unless you are paranoid about compatibility problems. The Python rules for bug fix releases are fairly strict and it is unlikely that a newer bug fix release will not work with ArcGIS.
What if I don't have ArcGIS?
We recommend the latest version of Python that is supported by GeoEco. At the time of this writing, it was Python 2.5.
Installation procedure
import sys
print 'Python ' + sys.version
print 'Press Enter to close this window...'
sys.stdin.readline()
The GeoEco setup program requires that the pywin32 Python package be installed for your version of Python. Pywin32 is also known as Python Extensions for Windows.
Python packages
Python packages are libraries of functions written in Python. When you install a package, you must choose the version of Python that you want it applied to. The setup program then installs the package into the directory for the Python release you chose.
Many Python packages will only work for specific releases of Python. For these, it is important that you download the package version that applies to your version of Python. The package download web page will usually list multiple versions of the package, with file names that only differ by Python version. Pywin32 is one of these. If you want to install it into your Python 2.4 installation, you must download pywin32-xxx.win32-py2.4.exe. To install it into your Python 2.5 installation, you must download pywin32-xxx.win32-py2.5.exe.
A common mistake is to install a package for one version of Python but not realize that your Windows file associations are configured to run another version. For example, you might have both Python 2.4 and 2.5 installed and then install pywin32 for Python 2.4, not remembering that Python 2.5 was the last version you installed and therefore the version configured in the Windows file assocation database. When you then run a Python script that requires pywin32, will execute under Python 2.5 but fail to find pywin32 because you installed the pywin32 for 2.4 not 2.5. It will seem as though your installation of pywin32 had no effect.
How Python packages appear in the Add/Remove Programs window
When you install a Python package, Windows adds it to the Add/Remove Programs window in the Control Panel (this window is called Programs and Features in Windows Vista). In this window, the package will typically be named "Python X.Y NAME-VERSION" where X and Y are the Python version the packaged was installed for, NAME is the name of the package, and VERSION is the package's version. Packages often use the same versioning scheme as Python, but some use different schemes.
The pywin32 package
This package exposes the functionality of the Windows operating system to Python scripts. GeoEco uses it for certain installation tasks and to expose its Python classes using Microsoft COM Automation, allowing virtually any scripting language can instantiate GeoEco classes and call their functions and properties.
Future releases of GeoEco for other operating systems will not require pywin32.
Installation procedure
If you installed a previous version of GeoEco, uninstall it now:
Download and install the GeoEco Python package from http://code.env.duke.edu/projects/mget/. Be sure to install the package that is specific to your version of Python.
Many MGET tools will be functional after you install the GeoEco Python package. But some tools rely on other software applications, libraries, and Python packages. To ease the burden of getting started, GeoEco requires almost none of these to be on your machine when you install it. All GeoEco tools check their software dependencies when you invoke them, and report error messages and brief installation instructions if required software is not installed.
If you prefer to get started right away, you can rely on these error messages to prompt you for additional software installations when they are needed. If you prefer not to see these error messages, you can preemptively install all of the additional software now.
The best way to learn MGET is to try it out. Here are some examples.
Please email Jason Roberts (jason.roberts@duke.edu) with any questions or feedback.
If you would like to observe the details of the processing performed by MGET tools, you can enable verbose logging.
The Reference Documentation is generated in several formats that are tailored to specific user communities:
Marine Geospatial Ecology Tools is built atop a lot of other software, much of it free. We would particularly like to thank these developers for making their excellent work freely reusable. Without your work, MGET would never have gotten off the ground. Cheers to all of you!
Development of Marine Geospatial Ecology Tools is funded by:
Except where otherwise noted, this document and the Marine Geospatial Ecology Tools software is Copyright © 2007 by Jason J. Roberts.
The terms "MGET" and "GeoEco" are synonymous with, and occasionally used instead of, "Marine Geospatial Ecology Tools".
MGET is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. MGET is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License (available in the file LICENSE.txt) for more details.
MGET makes use of several other programs graciously provided for free by other developers. MGET "aggregates" these under the terms of the GNU GPL. These programs require that their original license be reproduced when they are redistributed. Please see the file LICENSE.txt for the copyright notices and licensing details for these programs. Many thanks to these developers for their contributions.