Changeset 404

Timestamp:

06/08/09 13:48:16 (4 years ago)

Author:

jjr8

Message:

This and the previous few revisions fix and implement:

* #264: Predict GAM and Predict GLM tools should check the left and bottom extents and the number of rows and columns rather than all four extents
* #265: The Raster Type search option does not work for batch processing tools that find rasters
* #284: Tools that do not have "Run Python script in process" checked run extremely slowly under ArcGIS 9.3
* #357: Add support for ArcGIS 9.3.1
* #358: When provided with raster layers, Predict GAM and Predict GLM tools fail with GDAL Error 4: `<raster>' does not exist in the file system, and is not recognised as a supported dataset name.
* #359: With ArcGIS 9.3.1, Predict GAM and Predict GLM tools fail with RPy_RException: Error: GDAL Error 5: Access window out of range in RasterIO()
* #360: All tools that invoke R leak memory
* #361: Python's logging package causes large memory leaks in "in process" MGET tools under ArcGIS 9.3

The ArcGIS 9.3 toolbox still needs to rebuilt and more testing needs to happen, but this revision represents a large amount of the total work required for MGET 0.7b1.

Location:

MGET/Branches/Jason/PythonPackage/src/GeoEco

Files:

• ArcGIS.py (modified) (4 diffs)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2030.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2031.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2040.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2041.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2050.pyd (added)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2051.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2060.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2061.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2062.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2070.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2071.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2072.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2080.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2081.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python24/_rpy2090.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2030.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2031.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2040.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2041.pyd (deleted)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2050.pyd (added)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2051.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2060.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2061.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2062.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2070.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2071.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2072.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2080.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2081.pyd (modified) (previous)
• AssimilatedModules/rpy/lib/win32/Python25/_rpy2090.pyd (modified) (previous)
• AssimilatedModules/rpy/src-rpy-1.0.3/BUILDING_FOR_GEOECO.txt (modified) (1 diff)
• AssimilatedModules/rpy/src-rpy-1.0.3/src/io.c (modified) (2 diffs)
• AssimilatedModules/rpy/src-rpy-1.0.3/src/rpymodule.c (modified) (2 diffs)
• DataManagement/ArcGISRasters.py (modified) (3 diffs)
• DataManagement/Directories.py (modified) (1 diff)
• DataProducts/Aviso.py (modified) (5 diffs)
• DataProducts/NOAA/CoastWatchAVHRR.py (modified) (1 diff)
• Documentation/GettingStarted.xsl (modified) (3 diffs)
• R.py (modified) (4 diffs)
• Statistics/Exploratory.py (modified) (3 diffs)
• Statistics/Modeling.py (modified) (25 diffs)
• Statistics/PredictLMForArcGISRasters.r (modified) (1 diff)

MGET/Branches/Jason/PythonPackage/src/GeoEco/ArcGIS.py

@@ -933 +933 @@
         # turn on debug logging.)
 
-        if invokedFromArcGISApplication and GeoprocessorManager._ArcGISMajorVersion == 9 and GeoprocessorManager._ArcGISMinorVersion >= 3:
+        if invokedFromArcGISApplication and (GeoprocessorManager._ArcGISMajorVersion > 9 or GeoprocessorManager._ArcGISMajorVersion == 9 and GeoprocessorManager._ArcGISMinorVersion >= 3):
+            _ShutdownLoggingToAvoidArcGIS93MemoryLeak()
+            GeoprocessorManager.SetGeoprocessor(None)
             raise
         else:
@@ -940 +942 @@
     if caughtException:
         Logger.Debug(_(u'Python script %s exiting with return value 1, indicating failure.') % sys.argv[0])
+        GeoprocessorManager.SetGeoprocessor(None)
         sys.exit(1)
 
@@ -945 +948 @@
 
     Logger.Debug(_(u'Python script %s completed successfully.') % sys.argv[0])
+
+    if invokedFromArcGISApplication and (GeoprocessorManager._ArcGISMajorVersion > 9 or GeoprocessorManager._ArcGISMajorVersion == 9 and GeoprocessorManager._ArcGISMinorVersion >= 3):
+        _ShutdownLoggingToAvoidArcGIS93MemoryLeak()
+    GeoprocessorManager.SetGeoprocessor(None)
+
+def _ShutdownLoggingToAvoidArcGIS93MemoryLeak():
+
+    # Clean up objects allocated by the logging module that leak when
+    # tools are run "in process" under ArcGIS 9.3. For a detailed
+    # discussion, see MGET ticket #361.
+    
+    try:
+        logging.raiseExceptions = 0
+        logging.shutdown()
+        while len(logging._handlerList) > 0:
+            del logging._handlerList[0]
+        import atexit
+        for i in range(len(atexit._exithandlers)):
+            if atexit._exithandlers[i][0] == logging.shutdown:
+                del atexit._exithandlers[i]
+                break
+    except:
+        pass
 
 
@@ -1386 +1412 @@
                 geoprocessor = GeoprocessorManager.GetGeoprocessor()
                 if geoprocessor is not None:
-                    while i < geoprocessor.MessageCount:
-                        sev = geoprocessor.GetSeverity(i)
-                        if sev == 0:
-                            self._LogInfo(geoprocessor.GetMessage(i))
-                        elif sev == 1:
-                            self._LogWarning(geoprocessor.GetMessage(i))
-                        else:
-                            self._LogError(geoprocessor.GetMessage(i))
-                        i += 1
+                    try:
+                        while i < geoprocessor.MessageCount:
+                            sev = geoprocessor.GetSeverity(i)
+                            if sev == 0:
+                                self._LogInfo(geoprocessor.GetMessage(i))
+                            elif sev == 1:
+                                self._LogWarning(geoprocessor.GetMessage(i))
+                            else:
+                                self._LogError(geoprocessor.GetMessage(i))
+                            i += 1
+                    finally:
+                        del geoprocessor
             except:
                 pass

MGET/Branches/Jason/PythonPackage/src/GeoEco/AssimilatedModules/rpy/src-rpy-1.0.3/BUILDING_FOR_GEOECO.txt

@@ -57 +57 @@
 14. For each version of Python supported by GeoEco (assume version number is XY, e.g. 25):
 
-    a. Run: \pythonXY\python.exe setup.py build
+    a. Run: C:\pythonXY\python.exe setup.py build
     b. Run: copy build\lib.win32-X.Y\_rpy2081.pyd ..\lib\win32\PythonXY
     c. Add ..\lib\win32\PythonXY\_rpy2081.pyd to subversion (e.g. right click on the file, select TortoiseSVN --> Add)

MGET/Branches/Jason/PythonPackage/src/GeoEco/AssimilatedModules/rpy/src-rpy-1.0.3/src/io.c

@@ -192 +192 @@
 
   argformat = (char *)PyMem_Malloc((strlen(name)+3)*sizeof(char));
+  if (!argformat)
+    return PyErr_NoMemory();
   sprintf(argformat, "O:%s", name);
-  if (!PyArg_ParseTuple(args, argformat, &func))
-    return NULL;
-
+  if (!PyArg_ParseTuple(args, argformat, &func)) {
+    PyMem_Free(argformat);
+    PyErr_SetString(PyExc_ValueError, "Invalid arguments");
+    return NULL;
+  }
+  PyMem_Free(argformat);
+
+  Py_XDECREF(*var);
   Py_INCREF(func);
   *var = func;
@@ -224 +231 @@
 {
   if (o) {
+    Py_INCREF(o);
     return o;
   } else {

MGET/Branches/Jason/PythonPackage/src/GeoEco/AssimilatedModules/rpy/src-rpy-1.0.3/src/rpymodule.c

@@ -1779 +1779 @@
     R_RunExitFinalizers();  
     CleanEd();              
-#if (R_VERSION >= R_Version(2,7,0))
+#if (R_VERSION >= R_Version(2,5,0))
     Rf_KillAllDevices();
 #else
@@ -1842 +1842 @@
   use_numeric = i;
 
+/* jason.roberts@duke.edu: only initialize numpy the first time we are called; always succeed on subsequent calls*/
+  
+  if(first==1)
+    {
+      first=0;
 #ifdef WITH_NUMERIC
-  if(use_numeric)
-    init_numeric();
-#endif
-  
-  if(first==1)
-    {
-      first=0;
-      Py_INCREF(Py_None);
-      return Py_None;
-    }
-  else
-    {
-      PyErr_SetString(PyExc_RuntimeError, "Only one R object may be instantiated per session");
-      return NULL;
-    }
+      if(use_numeric)
+        init_numeric();
+#endif
+    }
+
+  Py_INCREF(Py_None);
+  return Py_None;
 }
 

MGET/Branches/Jason/PythonPackage/src/GeoEco/DataManagement/ArcGISRasters.py

@@ -74 +74 @@
         finally:
             cls.DeleteTemporaryRastersThatPreventedArcGISCrash(tempRasters)
+
+    @classmethod
+    def CopySilent(cls, sourceRaster, destinationRaster, overwriteExisting=False):
+        oldLogInfoAsDebug = Logger.GetLogInfoAsDebug()
+        Logger.SetLogInfoAsDebug(True)
+        try:
+            cls.Copy(sourceRaster, destinationRaster, overwriteExisting)
+        finally:
+            Logger.SetLogInfoAsDebug(oldLogInfoAsDebug)
 
     @classmethod
@@ -1167 +1176 @@
 
 AddArgumentMetadata(ArcGISRaster.Copy, u'sourceRaster',
-    typeMetadata=ArcGISRasterTypeMetadata(mustExist=True),
+    typeMetadata=ArcGISRasterLayerTypeMetadata(mustExist=True),
     description=_(u'Raster to copy.'),
     arcGISDisplayName=_(u'Source raster'))
@@ -1188 +1197 @@
 exists."""),
     initializeToArcGISGeoprocessorVariable=u'OverwriteOutput')
+
+# Public method: ArcGISRaster.CopySilent
+
+AddMethodMetadata(ArcGISRaster.CopySilent,
+    shortDescription=_(u'Copies an ArcGIS raster and logs a debug message rather than an informational message.'),
+    longDescription=_(
+u"""This method does the same thing as the Copy method, except it logs
+a debug message rather than an informational message. It is intended
+for use when the raster-copy operation is not imporant enough to
+warrent notifying the user (for example, when an output raster is
+extracted from a temporary directory to the final location)."""),
+    isExposedToPythonCallers=True,
+    isExposedByCOM=True,
+    dependencies=[ArcGISDependency(9, 1)])
+
+CopyArgumentMetadata(ArcGISRaster.Copy, u'cls', ArcGISRaster.CopySilent, u'cls')
+CopyArgumentMetadata(ArcGISRaster.Copy, u'sourceRaster', ArcGISRaster.CopySilent, u'sourceRaster')
+CopyArgumentMetadata(ArcGISRaster.Copy, u'destinationRaster', ArcGISRaster.CopySilent, u'destinationRaster')
+CopyArgumentMetadata(ArcGISRaster.Copy, u'overwriteExisting', ArcGISRaster.CopySilent, u'overwriteExisting')
 
 # Public method: ArcGISRaster.CreateTemporaryRastersToPreventArcGISCrash

MGET/Branches/Jason/PythonPackage/src/GeoEco/DataManagement/Directories.py

@@ -151 +151 @@
 
                 gp = GeoprocessorManager.GetWrappedGeoprocessor()
-                if gp is not None and isinstance(gp.ScratchWorkspace, basestring) and len(gp.ScratchWorkspace) > 0 and os.path.isdir(gp.ScratchWorkspace):
+                if gp is not None and hasattr(gp, 'ScratchWorkspace') and isinstance(gp.ScratchWorkspace, basestring) and len(gp.ScratchWorkspace) > 0 and os.path.isdir(gp.ScratchWorkspace):
                     parentDirectory = os.path.join(gp.ScratchWorkspace, u'GeoEcoTemp')
 

MGET/Branches/Jason/PythonPackage/src/GeoEco/DataProducts/Aviso.py

@@ -21 +21 @@
 import datetime
 import os
+import time
 
 from GeoEco.DataManagement.ArcGISRasters import ArcGISRaster
@@ -153 +154 @@
             coordinateSystem = "PROJCS['Sphere_Mercator',GEOGCS['GCS_Sphere',DATUM['D_Sphere',SPHEROID['Sphere',6371000.0,0.0]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Mercator'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',180.0],PARAMETER['Standard_Parallel_1',0.0],UNIT['Meter',1.0]]"
 
-        # Look up the minimum longitude and latitude and the step
-        # size. All of these are in degrees.
+        # Look up the minimum longitude and latitude, max latitude,
+        # and the step size (all in degrees).
 
         if not ds.has_key('LatLonMin'):
@@ -163 +164 @@
         minLon = ds['LatLonMin'][1]
         minLat = ds['LatLonMin'][0]
+        maxLat = max(ds['NbLatitudes'][:])
         step = ds['LatLonStep'][0]
 
         Logger.Debug(_(u'Minimum longitude = %s (on 0-to-360 coordinate system)') % str(minLon))
         Logger.Debug(_(u'Minimum latitude = %s') % str(minLat))
+        Logger.Debug(_(u'Maximum latitude = %s') % str(maxLat))
         Logger.Debug(_(u'Cell size = %s degrees') % str(step))
 
@@ -393 +396 @@
                 # Download the image as a numpy array.
 
-                image = ds[gridNames[j]][startIndex + i]
+                retries = 0
+                while True:
+                    try:
+                        image = ds[gridNames[j]][startIndex + i]
+                    except Exception, e:
+                        if retries < 3 and isinstance(e, ValueError) and str(e).lower() in ['substring not found', 'string length not a multiple of item size']:
+                            if retries == 0:
+                                Logger.Warning(_(u'Failed to download %(grid)s for %(date)s (time slice %(slice)i) from the Aviso OPeNDAP server. Sleeping for 5 seconds and retrying the download (retry 1 of 3). Error details: the pydap library reported %(error)s: %(msg)s') % {u'grid': gridNames[j], u'date': str(availableDates[i]), u'slice': startIndex + i, u'error': e.__class__.__name__, u'msg': unicode(e)})
+                                time.sleep(5)
+                            else:
+                                Logger.Warning(_(u'Failed to download %(grid)s for %(date)s (time slice %(slice)i) from the Aviso OPeNDAP server. Sleeping for 60 seconds and retrying the download (retry %(ret)i of 3). Error details: the pydap library reported %(error)s: %(msg)s') % {u'grid': gridNames[j], u'date': str(availableDates[i]), u'slice': startIndex + i, u'error': e.__class__.__name__, u'msg': unicode(e), u'ret': retries+1})
+                                time.sleep(60)
+                            retries += 1
+                        else:
+                            Logger.RaiseException(RuntimeError(_(u'Failed to download %(grid)s for %(date)s (time slice %(slice)i) from the Aviso OPeNDAP server. The download was attempted four times without success. Most likely, a temporary network problem is affecting communication with the server. Please try again later. If this problem continues to occur, contact the MGET development team for more help. Error details: the pydap library reported %(error)s: %(msg)s') % {u'grid': gridNames[j], u'date': str(availableDates[i]), u'slice': startIndex + i, u'error': e.__class__.__name__, u'msg': unicode(e)}))
+                    else:
+                        break
 
                 # Transpose and flip the image, so it is in the
@@ -423 +442 @@
                 # expected map extent. If so, strip it too.
 
-                if isGlobal and max(ds['NbLatitudes'][:]) == 90.:
+                if isGlobal and maxLat == 90.:
                     image = image[1:, :]
 

MGET/Branches/Jason/PythonPackage/src/GeoEco/DataProducts/NOAA/CoastWatchAVHRR.py

@@ -81 +81 @@
         
         if self.GetBinariesPath() is None:
-            Logger.RaiseException(SoftwareNotInstalledError(_(u'This function requires the NOAA CoastWatch Utilities version 3.2 or later. Download and install the Utilities from http://coastwatch.noaa.gov/cw_software.html. Be sure you enable the Command Line Tools option; you must explicitly it, as it is disabled by default. Also, if you elect to install the Utilities to a directory other than the default, after the installation is complete you must add the path to the Utilities\' "bin" directory to the PATH environment variable by following the instructions in the "Installation Notes" section of the CoastWatch Utilities User\'s Guide.')))
+            Logger.RaiseException(SoftwareNotInstalledError(_(u'This function requires the NOAA CoastWatch Utilities version 3.2 or later. Download and install the Utilities from http://coastwatch.noaa.gov/cwn/cw_software.html. Be sure you enable the Command Line Tools option; you must explicitly it, as it is disabled by default. Also, if you elect to install the Utilities to a directory other than the default, after the installation is complete you must add the path to the Utilities\' "bin" directory to the PATH environment variable by following the instructions in the "Installation Notes" section of the CoastWatch Utilities User\'s Guide.')))
 
         if sys.platform == 'win32' and not os.path.isfile(os.path.join(CoastWatchUtilitiesDependency._BinariesPath, u'cwinfo.exe')) or sys.platform != 'win32' and not os.path.isfile(os.path.join(CoastWatchUtilitiesDependency._BinariesPath, u'cwinfo')):
-            Logger.RaiseException(SoftwareNotInstalledError(_(u'This function requires the NOAA CoastWatch Utilities version 3.2 or later. The Utilities appear to be installed, but when the installation was performed, the Command Line Tools option was not enabled. You must reinstall the Utilities with this option enabled. First uninstall your existing version. Then download the Utilities from http://coastwatch.noaa.gov/cw_software.html and install them. Be sure you enable the Command Line Tools option; you must explicitly it, as it is disabled by default. Also, if you elect to install the Utilities to a directory other than the default, after the installation is complete you must add the path to the Utilities\' "bin" directory to the PATH environment variable by following the instructions in the "Installation Notes" section of the CoastWatch Utilities User\'s Guide.')))
+            Logger.RaiseException(SoftwareNotInstalledError(_(u'This function requires the NOAA CoastWatch Utilities version 3.2 or later. The Utilities appear to be installed, but when the installation was performed, the Command Line Tools option was not enabled. You must reinstall the Utilities with this option enabled. First uninstall your existing version. Then download the Utilities from http://coastwatch.noaa.gov/cwn/cw_software.html and install them. Be sure you enable the Command Line Tools option; you must explicitly it, as it is disabled by default. Also, if you elect to install the Utilities to a directory other than the default, after the installation is complete you must add the path to the Utilities\' "bin" directory to the PATH environment variable by following the instructions in the "Installation Notes" section of the CoastWatch Utilities User\'s Guide.')))
 
         CoastWatchUtilitiesDependency._CommandLineUtilitiesInstalled = True

MGET/Branches/Jason/PythonPackage/src/GeoEco/Documentation/GettingStarted.xsl

@@ -1 @@
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 
 <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
@@ -456 +456 @@
       consult an ArcGIS expert for more information.</li>
 
-      <li style="margin-top: 1em;"><a href="http://www.r-project.org/"><b>R statistics program</b></a>, version 2.5.1 to 2.9.0 -
+      <li style="margin-top: 1em;"><a href="http://www.r-project.org/"><b>R statistics program</b></a>, version 2.5.0 to 2.9.0 -
       this is required by GeoEco tools that perform statistical calculations. You
       must have a version in the range indicated. Newer or older versions will not
@@ -468 +468 @@
       do not have MATLAB, download the MATLAB Component Runtime from the link above.</li>
 
-      <li style="margin-top: 1em;"><a href="http://coastwatch.noaa.gov/cw_cwfv3.html"><b>NOAA CoastWatch Utilities</b></a> -
+      <li style="margin-top: 1em;"><a href="http://coastwatch.noaa.gov/cwn/cw_software.html"><b>NOAA CoastWatch Utilities</b></a>, version 3.2 or later -
       these are required if you want to use any of the GeoEco tools that process
       CoastWatch data. In the CoastWatch Utilities setup, be sure to you enable the

MGET/Branches/Jason/PythonPackage/src/GeoEco/R.py

@@ -106 +106 @@
 
         r = cls.GetInterpreter()
-        if variableNames is not None:
-            for i in range(len(variableNames)):
-                if isinstance(variableValues[i], (datetime.datetime, datetime.date)) and timeZone is not None:
-                    r[variableNames[i]] = RInterpreter.PreconvertPythonObjectToRObject(variableValues[i], timeZone=timeZone)
-                else:
-                    r[variableNames[i]] = variableValues[i]
-
-        # If the caller passed us tables to load as data frames, load
-        # them now.
-
-        if tables is not None:
-            for i in range(len(tables)):
-                cls.LoadDataFrameFromArcGISTable(tables[i], dataFrameNamesForTables[i], xColumnName=xColumnName, yColumnName=yColumnName, zColumnName=zColumnName, mColumnName=mColumnName, stringsAsFactors=stringsAsFactors, timeZone=timeZone)
-
-        # Evaluate the caller's statements and return the last one.
-        
-        oldTracebackLogLevel = RInterpreter.GetLogTracebackAsInfo()
-        RInterpreter.SetLogTracebackAsInfo(True)
         try:
-            for i in range(len(statements)):
+            if variableNames is not None:
+                for i in range(len(variableNames)):
+                    if isinstance(variableValues[i], (datetime.datetime, datetime.date)) and timeZone is not None:
+                        r[variableNames[i]] = RInterpreter.PreconvertPythonObjectToRObject(variableValues[i], timeZone=timeZone)
+                    else:
+                        r[variableNames[i]] = variableValues[i]
+
+            # If the caller passed us tables to load as data frames, load
+            # them now.
+
+            if tables is not None:
+                for i in range(len(tables)):
+                    cls.LoadDataFrameFromArcGISTable(tables[i], dataFrameNamesForTables[i], xColumnName=xColumnName, yColumnName=yColumnName, zColumnName=zColumnName, mColumnName=mColumnName, stringsAsFactors=stringsAsFactors, timeZone=timeZone)
+
+            # Evaluate the caller's statements and return the last one.
+            
+            oldTracebackLogLevel = RInterpreter.GetLogTracebackAsInfo()
+            RInterpreter.SetLogTracebackAsInfo(True)
+            try:
+                for i in range(len(statements)):
+                    try:
+                        result = r(statements[i])
+                    except:
+                        if len(statements) > 1:
+                            Logger.Error(_(u'The failure occurred in statement %(num)i of %(total)i.') % {u'num': i+1, u'total': len(statements)})
+                        raise
+            finally:
+                RInterpreter.SetLogTracebackAsInfo(oldTracebackLogLevel)
+
+        # If the caller requested that we clear the global
+        # environment, do it now.
+
+        finally:
+            if clearGlobalEnv:
                 try:
-                    result = r(statements[i])
-                except:
-                    if len(statements) > 1:
-                        Logger.Error(_(u'The failure occurred in statement %(num)i of %(total)i.') % {u'num': i+1, u'total': len(statements)})
-                    raise
-        finally:
-            RInterpreter.SetLogTracebackAsInfo(oldTracebackLogLevel)
+                    r('rm(list=ls())')
+                finally:
+                    pass
+
+        # Return the result of the last statement.
             
         return result
@@ -418 +431 @@
         
         if major is None:
-            Logger.RaiseException(SoftwareNotInstalledError(_(u'This function requires R %i.%i.%i or a later version, but it is not installed. Please download and install a compatible version from http://www.r-project.org/.') % (self.MinimumMajorVersion, self.MinimumMinorVersion, self.MinimumPatchVersion)))
+            Logger.RaiseException(SoftwareNotInstalledError(_(u'This tool requires R %i.%i.%i or a later version, but it is not installed. Please download and install a compatible version from http://www.r-project.org/.') % (self.MinimumMajorVersion, self.MinimumMinorVersion, self.MinimumPatchVersion)))
             
         if self.MinimumMajorVersion > major or self.MinimumMajorVersion == major and self.MinimumMinorVersion > minor or self.MinimumMajorVersion == major and self.MinimumMinorVersion == minor and self.MinimumPatchVersion > patch:
-            Logger.RaiseException(SoftwareNotInstalledError(_(u'This function requires R %i.%i.%i or a later version, but verison %i.%i.%i is installed. Please upgrade R to a compatible version. You may download newer versions from http://www.r-project.org/.') % (self.MinimumMajorVersion, self.MinimumMinorVersion, self.MinimumPatchVersion, major, minor, patch)))
+            Logger.RaiseException(SoftwareNotInstalledError(_(u'This tool requires R %i.%i.%i or a later version, but version %i.%i.%i is installed. Please upgrade R to a compatible version. You may download newer versions from http://www.r-project.org/.') % (self.MinimumMajorVersion, self.MinimumMinorVersion, self.MinimumPatchVersion, major, minor, patch)))
+
+        (maxMajor, maxMinor, maxPatch) = self.GetGeoEcoMaxSupportedVersion()
+
+        if major > maxMajor or major == maxMajor and minor > maxMinor or major == maxMajor and minor == maxMinor and patch > maxPatch:
+            Logger.RaiseException(SoftwareNotInstalledError(_(u'This tool requires R, but the version of R that is installed (%i.%i.%i) is not yet supported by this tool. Please remove this version of R, install a compatible version (%i.%i.%i through %i.%i.%i), and try again. You can download R from http://www.r-project.org/. Alternatively, visit the MGET website to see if a new version of MGET is available that supports your version of R. If not, contact the MGET development team and request that support be added.') % (major, minor, patch, self.MinimumMajorVersion, self.MinimumMinorVersion, self.MinimumPatchVersion, maxMajor, maxMinor, maxPatch)))
 
         # If we haven't import the rpy module yet, import it. rpy
@@ -522 +540 @@
 
             rpy.set_default_mode(rpy.PROC_CONVERSION)
+            rpy.proc_table.clear()
             rpy.proc_table[lambda o: True] = RDependency._ConvertRObjectToPythonObject
 
@@ -559 +578 @@
 
                 rpy.r('.libPaths("%s")' % RDependency._PerUserPackageDir.replace('\\', '\\\\'))
+
+    _GeoEcoMaxSupportedRMajorVersion = None
+    _GeoEcoMaxSupportedRMinorVersion = None
+    _GeoEcoMaxSupportedRPatchVersion = None
+
+    @classmethod
+    def GetGeoEcoMaxSupportedVersion(cls):
+        
+        # Return immediately if we already obtained the version numbers.
+        
+        if RDependency._GeoEcoMaxSupportedRMajorVersion is not None:
+            return (RDependency._GeoEcoMaxSupportedRMajorVersion, RDependency._GeoEcoMaxSupportedRMinorVersion, RDependency._GeoEcoMaxSupportedRPatchVersion)
+
+        # Enumerate the _rpy*.pyd files in the rpy directory.
+
+        import GeoEco
+        rpyDir = os.path.join(os.path.dirname(sys.modules[u'GeoEco'].__file__), 'AssimilatedModules', 'rpy')
+
+        import glob
+        if sys.platform.lower() == 'win32':
+            extensionModules = glob.glob(os.path.join(rpyDir, '_rpy[0123456789][0123456789][0123456789][0123456789].pyd'))
+        else:
+            Logger.RaiseException(NotImplementedError('Tools that use the R statistical program are not currently supported on non-Win32 platforms.'))
+
+        # Extract the version number from the last file.
+        
+        extensionModules.sort()
+        
+        major = int(extensionModules[-1][-8:-7])
+        minor = int(extensionModules[-1][-7:-5])
+        patch = int(extensionModules[-1][-5:-4])
+
+        RDependency._GeoEcoMaxSupportedRMajorVersion = major
+        RDependency._GeoEcoMaxSupportedRMinorVersion = minor
+        RDependency._GeoEcoMaxSupportedRPatchVersion = patch
+
+        Logger.Debug(_(u'The latest version of R supported by this version of GeoEco is R %i.%i.%i.') % (major, minor, patch))
+
+        # Return successfully.
+        
+        return (major, minor, patch)
 
     _RMajorVersion = None

MGET/Branches/Jason/PythonPackage/src/GeoEco/Statistics/Exploratory.py

@@ -61 +61 @@
                 r('%s <- NULL' % transformsName)
 
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'ScatterplotMatrixForDataframe.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'ScatterplotMatrixForDataframe.r'), False)
             if outputFile is None:
                 r('ScatterplotMatrix(%s, transforms=%s, .diag="%s", lower="%s", upper="%s")' % (dataFrameName, transformsName, str(diagonal), str(lower), str(upper)))
@@ -99 +99 @@
                     Logger.RaiseException(ValueError(_(u'The table %(table)s is empty.') % {u'table': table}))
             
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'DensityHistogramForDataframe.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'DensityHistogramForDataframe.r'), False)
 
             if transform is None:
@@ -175 +175 @@
                     Logger.RaiseException(ValueError(_(u'The table %(table)s is empty.') % {u'table': table}))
             
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'DensityHistogramForDataframe.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'DensityHistogramForDataframe.r'), False)
 
             if transform is None:

MGET/Branches/Jason/PythonPackage/src/GeoEco/Statistics/Modeling.py

@@ -57 +57 @@
         
         try:
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'SummarizeModel.r'))
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'FitGLMForDataframe.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'SummarizeModel.r'), False)
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'FitGLMForDataframe.r'), False)
 
             tempDir = TemporaryDirectory()
@@ -134 +134 @@
         
         try:
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'SummarizeModel.r'))
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'FitGAMForDataframe.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'SummarizeModel.r'), False)
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'FitGAMForDataframe.r'), False)
 
             tempDir = TemporaryDirectory()
@@ -207 +207 @@
             # Log a summery of the model, to remind the user what it is.
 
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'SummarizeModel.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'SummarizeModel.r'), False)
 
             r('writeLines("")')
@@ -233 +233 @@
             # no way to suppress these messages from within R.
 
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'BayesPredictGAMForArcGISRasters.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'BayesPredictGAMForArcGISRasters.r'), False)
 
             oldLoggingLevel = r.LogInfoAsDebug
@@ -260 +260 @@
                     gp.BuildPyramids_Management(tempOutputRaster)
 
-            # Move the output rasters to the requested destinations.
+            # Copy the output rasters to the requested destinations.
 
             for i in range(len(outputProbabilityRasters)):
-                ArcGISRaster.MoveSilent(os.path.join(tempDir.Path, u'output%i' % i), outputProbabilityRasters[i], overwriteExisting=overwriteExisting)
+                ArcGISRaster.CopySilent(os.path.join(tempDir.Path, u'output%i' % i), outputProbabilityRasters[i], overwriteExisting=overwriteExisting)
 
         # Delete R variables assigned by this function.
@@ -315 +315 @@
             # Create the plot.
         
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'PerfPlotForBinaryModel.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'PerfPlotForBinaryModel.r'), False)
 
             if measure2 is None:
@@ -398 +398 @@
             # Create the plot.
         
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'ROCPlotForBinaryModel.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[cls.__module__].__file__), 'ROCPlotForBinaryModel.r'), False)
 
             if cutoffValue is None:
@@ -832 +832 @@
         # Log a summery of the model, to remind the user what it is.
 
-        R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'SummarizeModel.r'))
+        R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'SummarizeModel.r'), False)
 
         r('writeLines("")')
@@ -885 +885 @@
             Logger.Info(_(u'Predicting...'))
 
-            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'PredictLMForArcGISRasters.r'))
+            R.EvaluateFile(os.path.join(os.path.dirname(sys.modules[GLM.__module__].__file__), 'PredictLMForArcGISRasters.r'), False)
 
             oldLoggingLevel = r.LogInfoAsDebug
@@ -926 +926 @@
         # Move the output rasters to the requested destinations.
 
-        ArcGISRaster.MoveSilent(tempResponseRaster, outputResponseRaster, overwriteExisting=overwriteExisting)
+        ArcGISRaster.CopySilent(tempResponseRaster, outputResponseRaster, overwriteExisting=overwriteExisting)
         
         if outputErrorRaster is not None:
-            ArcGISRaster.MoveSilent(tempErrorRaster, outputErrorRaster, overwriteExisting=overwriteExisting)
+            ArcGISRaster.CopySilent(tempErrorRaster, outputErrorRaster, overwriteExisting=overwriteExisting)
             
         if outputBinaryResponseRaster is not None:
-            ArcGISRaster.MoveSilent(tempBinaryRaster, outputBinaryResponseRaster, overwriteExisting=overwriteExisting)
+            ArcGISRaster.CopySilent(tempBinaryRaster, outputBinaryResponseRaster, overwriteExisting=overwriteExisting)
 
     # Delete R variables assigned by this function.
@@ -995 +995 @@
         rastersForPredictors[r['yVar']] = yRaster
 
-    # Check each predictor raster. If it has a different coordinate
-    # system, extent, or cell size than that of the template raster,
-    # project it in the temp directory.
-    #
-    # Note that ArcGIS does not always seem to represent the cell size
-    # and extent of rasters at the fullest possible precision, so we
-    # perform comparisons at less than full precision.
-
-    Logger.Info('Checking projections, extents, and cell sizes of predictor rasters and reprojecting and clipping as needed...')
+    # Check each predictor raster to see if it needs to be copied,
+    # projected or clipped before we can use it.
+
+    Logger.Info(_('Checking coordinate systems, extents, and cell sizes of predictor rasters and reprojecting and clipping as needed to make them conform to the template raster...'))
 
     from GeoEco.Types import EnvelopeTypeMetadata
@@ -1010 +1005 @@
     vars = copy.deepcopy(rastersForPredictors.keys())           # I'm going to modify the dictionary in the loop below; not sure if deepcopy is needed, but playing it safe.
     rasters = copy.deepcopy(rastersForPredictors.values())
-    arc93SnapRaster = None
 
     for i in range(len(vars)):
-        if rasters[i] == templateRaster:
-            continue
-        
+        needToCopy = False
         needToProject = False
         needToClip = False
+
+        # If this is predictor raster is a raster layer, we need to
+        # make a copy of it in the file system so that it can be read
+        # by the R code using the rgdal package.
+        
         d = gp.Describe(rasters[i])
-        cs = gp.CreateSpatialReference_management(d.SpatialReference).split(u';')[0]
-
-        if cs != coordinateSystem:
-            Logger.Debug(_(u'%(raster)s must be projected because its coordinate system (%(cs1)s) does not match the output coordinate system (%(cs2)s.') % {u'raster': rasters[i], u'cs1': cs, u'cs2': coordinateSystem})
-            Logger.Info(_(u'Projecting and clipping %(raster)s because its coordinate system does not match that of the template raster.') % {u'raster': rasters[i]})
-            needToProject = True
-
-        else:
-            [rasterLeft, rasterBottom, rasterRight, rasterTop] = EnvelopeTypeMetadata.ParseCoordinatesFromString(d.Extent)
-
-            if rasterLeft > templateLeft:
-                Logger.RaiseException(_(u'The template raster is too large; its left edge falls outside the left edge of the predictor raster %(raster)s. Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i]})
-            if rasterRight < templateRight:
-                Logger.RaiseException(_(u'The template raster is too large; its right edge falls outside the right edge of the predictor raster %(raster)s. Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i]})
-            if rasterTop > templateTop:
-                Logger.RaiseException(_(u'The template raster is too large; its top edge falls outside the top edge of the predictor raster %(raster)s. Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i]})
-            if rasterBottom < templateBottom:
-                Logger.RaiseException(_(u'The template raster is too large; its bottom edge falls outside the bottom edge of the predictor raster %(raster)s. Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i]})
-                
+        if d.DataType.lower() == u'rasterlayer':
+            needToCopy = True
+
+        # If this is not the template raster, check to see whether it
+        # needs to be projected or clipped.
+
+        if rasters[i] != templateRaster:
+            cs = gp.CreateSpatialReference_management(d.SpatialReference).split(u';')[0]
+
+            if cs.lower() != coordinateSystem.lower():
+                Logger.Info(_(u'Projecting and clipping %(raster)s. Its coordinate system does not match that of the template raster.') % {u'raster': rasters[i]})
+                needToProject = True
+
             elif abs(1.0 - cellSize / d.MeanCellWidth) > 0.00001:
-                Logger.Debug(_(u'%(raster)s must be projected because its cell size (%(cs1)f) does not match the output cell size (%(cs2)f).') % {u'raster': rasters[i], u'cs1': d.MeanCellWidth, u'cs2': cellSize})
-                Logger.Info(_(u'Projecting and clipping %(raster)s because its cell size does not match that of the template raster.') % {u'raster': rasters[i]})
+                Logger.Info(_(u'Projecting and clipping %(raster)s. Its cell size (%(cs1)s) does not match the cell size of the template raster (%(cs2)s).') % {u'raster': rasters[i], u'cs1': repr(d.MeanCellWidth), u'cs2': repr(cellSize)})
                 needToProject = True
 
             else:
+                [rasterLeft, rasterBottom, rasterRight, rasterTop] = EnvelopeTypeMetadata.ParseCoordinatesFromString(d.Extent)
+                    
                 deltaLeft = (float(extent.split()[0]) - float(d.Extent.split()[0])) / cellSize     # Difference, in cells, between the left edge of the template and this raster
                 deltaBottom = (float(extent.split()[1]) - float(d.Extent.split()[1])) / cellSize   # Difference, in cells, between the bottom edge of the template and this raster
@@ -1050 +1042 @@
 
                 if deltaLeft < -0.01:
-                    Logger.RaiseException(_(u'The template raster is too large. The left edge of the predictor raster %(raster)s is inside the left edge of the template raster. Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i]})
-                if deltaRight > -0.01:
-                    Logger.RaiseException(_(u'The template raster is too large. The left edge of the predictor raster %(raster)s is inside the left edge of the template raster. Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i]})
-
-                if deltaLeft < 0.01 and deltaBottom < 0.01 and deltaRight < 0.01 and deltaTop < 0.01:
-                    Logger.Debug(_(u'%(raster)s has the same coordinate system, cell size, and extent as the template raster.') % {u'raster': rasters[i]})
+                    Logger.RaiseException(ValueError(_(u'The template raster is too large; its left edge falls (%(e1)s) outside the left edge of the predictor raster %(raster)s (%(e2)s). Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i], u'e1': repr(templateLeft), u'e2': repr(rasterLeft)}))
+                if deltaRight > 0.01:
+                    Logger.RaiseException(ValueError(_(u'The template raster is too large; its right edge falls (%(e1)s) outside the right edge of the predictor raster %(raster)s (%(e2)s). Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i], u'e1': repr(templateRight), u'e2': repr(rasterRight)}))
+                if deltaTop > 0.01:
+                    Logger.RaiseException(ValueError(_(u'The template raster is too large; its top edge falls (%(e1)s) outside the top edge of the predictor raster %(raster)s (%(e2)s). Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i], u'e1': repr(templateTop), u'e2': repr(rasterTop)}))
+                if deltaBottom < -0.01:
+                    Logger.RaiseException(ValueError(_(u'The template raster is too large; its bottom edge falls (%(e1)s) outside the bottom edge of the predictor raster %(raster)s (%(e2)s). Please reduce the extent of the template raster such that it does not exceed the extents of any of the predictor rasters.') % {u'raster': rasters[i], u'e1': repr(templateBottom), u'e2': repr(rasterBottom)}))
+
+                if abs(deltaLeft) < 0.01 and abs(deltaBottom) < 0.01 and abs(deltaRight) < 0.01 and abs(deltaTop) < 0.01 and d.Width == templateDescribe.Width and d.Height == templateDescribe.Height:
+                    Logger.Info(_(u'%(raster)s has the same coordinate system, cell size, and extent as the template raster.') % {u'raster': rasters[i]})
 
                 elif abs(deltaLeft - round(deltaLeft)) < 0.01 and abs(deltaBottom - round(deltaBottom)) < 0.01 and abs(deltaRight - round(deltaRight)) < 0.01 and abs(deltaTop - round(deltaTop)) < 0.01:
-                    Logger.Debug(_(u'%(raster)s must be clipped because it has the same coordinate system and cell size as the template raster, and is aligned on the same grid coordinates, but has a different extent.') % {u'raster': rasters[i]})
-                    Logger.Info(_(u'Clipping %(raster)s to the extent of the template raster.') % {u'raster': rasters[i]})
+                    Logger.Info(_(u'Clipping %(raster)s. Although it has the same coordinate system and cell size as the template raster and is aligned on the same grid coordinates, it has a different extent.') % {u'raster': rasters[i]})
                     needToClip = True
 
                 else:
-                    Logger.Debug(_(u'%(raster)s must be clipped because it has the same coordinate system and cell size as the template raster, and is aligned on the same grid coordinates, but has a different extent.') % {u'raster': rasters[i]})
-                    Logger.Info(_(u'Clipping %(raster)s to the extent of the template raster.') % {u'raster': rasters[i]})
+                    Logger.Info(_(u'Projecting and clipping %(raster)s. Although it has the same coordinate system and cell size as the template raster, it is aligned on different grid coordinates and has a different extent.') % {u'raster': rasters[i]})
                     needToProject = True
-                    
-
-                if abs(offsetX) > 0.01 or abs(offsetY) > 0.01:
-                    if abs(offsetX - round(offsetX)) > 0.01 or abs(offsetX - round(offsetX)) > 0.01:
-                        Logger.Debug(_(u'%(raster)s must be projected because its cell size (%(cs1)f) does not match the output cell size (%(cs2)f).') % {u'raster': rasters[i], u'cs1': d.MeanCellWidth, u'cs2': cellSize})
-                        Logger.Info(_(u'Projecting and clipping %(raster)s because its cell size does not match that of the template raster.') % {u'raster': rasters[i]})
-                        needToProject = True
+
+        # If this raster does not need to be copied, projected, or
+        # clipped, go on to the next one.
+
+        if not (needToCopy or needToProject or needToClip):
+            continue
+
+        # If we need to project, do it now. This will also take care
+        # of copying and clipping the raster, if needed.
 
         if needToProject:
 
-            # Use the caller's resampling technique, if
-            # one was provided.
+            # Use the caller's resampling technique, if one was
+            # provided. Otherwise use BILINEAR if it is a
+            # floating-point raster, or NEAREST if it is not.
             
-            resamplingTechnique = u'BILINEAR'
             if resamplingTechniques is not None and vars[i] in variableNames and variableNames.index(vars[i]) < len(resamplingTechniques):
-                resamplingTechnique= resamplingTechniques[variableNames.index(vars[i])]
-
-            # If this is ArcGIS 9.3, we need to create a snap raster
-            # with the caller's requested output extent and cell size.
-            # If we already created an X or Y coordinate raster, reuse
-            # it. Otherwise create one. Note that the values of the
-            # snap raster don't matter, just the extent and cell size.
-            # We call ArcGISRaster.CreateXRaster just because its an
-            # easy way to create a raster.
-
-            if (GeoprocessorManager.GetArcGISMajorVersion() > 9 or GeoprocessorManager.GetArcGISMajorVersion() == 9 and GeoprocessorManager.GetArcGISMinorVersion() >= 3) and arc93SnapRaster is None:
-                if createXRaster:
-                    arc93SnapRaster = xRaster
-                elif createYRaster:
-                    arc93SnapRaster = yRaster
-                else:
-                    arc93SnapRaster = os.path.join(tempDir.Path, u'snapraster')
-                    oldLogInfoAsDebug = Logger.GetLogInfoAsDebug()
-                    Logger.SetLogInfoAsDebug(True)
-                    try:
-                        ArcGISRaster.CreateXRaster(arc93SnapRaster, extent, cellSize, u'Center', coordinateSystem)
-                    finally:
-                        Logger.SetLogInfoAsDebug(oldLogInfoAsDebug)
+                resamplingTechnique = resamplingTechniques[variableNames.index(vars[i])]
+            elif d.PixelType[0] in ['F', 'f']:
+                resamplingTechnique = u'BILINEAR'
+            else:
+                resamplingTechnique = u'NEAREST'
 
             # Set the geoprocessor's Extent environment
@@ -1117 +1095 @@
             try:
                 # If this is ArcGIS 9.3, set the SnapRaster
-                # environment variable as well.
+                # environment variable to the template raster.
                 
                 if GeoprocessorManager.GetArcGISMajorVersion() > 9 or GeoprocessorManager.GetArcGISMajorVersion() == 9 and GeoprocessorManager.GetArcGISMinorVersion() >= 3:
@@ -1123 +1101 @@
                     if oldSnapRaster is None:
                         oldSnapRaster = u''
-                    gp.SnapRaster = arc93SnapRaster
+                    gp.SnapRaster = templateRaster
 
                 # Project the raster. If this is ArcGIS 9.3 or later,
@@ -1139 +1117 @@
                     else:
                         gp.ProjectRaster_management(rasters[i], tempRaster, coordinateSystem, resamplingTechnique, cellSize, None, extent.rsplit(' ', 2)[0])
+
+                    # Even though we provided an extent and cell size
+                    # of an existing raster (the template raster),
+                    # using values we obtained through ArcGIS APIs,
+                    # the ProjectRaster tool will not necessarily
+                    # create a raster that has exactly that extent. It
+                    # may be one cell larger or smaller in any of the
+                    # four directions. Different versions of ArcGIS
+                    # seem to work differently in this respect. It is
+                    # probably due to whether it uses 32-bit or 64-bit
+                    # floats, and other various floating-point
+                    # rounding issues.
+                    #
+                    # The prediction code in R requires that all
+                    # predictor rasters have the same extent (really,
+                    # the same number of rows and columns). If ArcGIS
+                    # produced a raster that was too small or large in
+                    # any dimension, try ProjectRaster again with
+                    # values that will hopefully yield what we want.
+
+                    describeTemp = gp.Describe(tempRaster)
+                    newEdges = _CheckResultingEdges(tempRaster, describeTemp.Extent, templateLeft, templateBottom, templateRight, templateTop, cellSize)
+
+                    if newEdges is not None:
+                        Logger.Debug(_(u'Reprojecting because the first attempt produced a raster with an extent (%(e1)s) that did not match the extent of the template raster (%(e2)s).') % {u'e1': describeTemp.Extent, u'e2': extent})
+                        gp.Extent = u' '.join(map(unicode, newEdges))
+                        tempRaster = os.path.join(tempDir.Path, u'projected%ia' % i)
+                        if GeoprocessorManager.GetArcGISMajorVersion() > 9 or GeoprocessorManager.GetArcGISMajorVersion() == 9 and GeoprocessorManager.GetArcGISMinorVersion() >= 3:
+                            gp.ProjectRaster_management(rasters[i], tempRaster, coordinateSystem, resamplingTechnique, cellSize)
+                        else:
+                            gp.ProjectRaster_management(rasters[i], tempRaster, coordinateSystem, resamplingTechnique, cellSize, None, extent.rsplit(' ', 2)[0])
+
+                        # Check the extent of the new attempt. If we
+                        # still did not get it, report an error.
+
+                        describeTemp = gp.Describe(tempRaster)
+                        [projectedLeft, projectedBottom, projectedRight, projectedTop] = EnvelopeTypeMetadata.ParseCoordinatesFromString(describeTemp.Extent)
+
+                        if projectedLeft != templateLeft or projectedBottom != templateBottom or projectedRight != templateRight or projectedTop != templateTop:
+                            Logger.RaiseException(RuntimeError(_(u'This tool was unable to project %(raster)s to the coordinate system of the template raster and obtain a projected raster with an extent that matched that of the template raster. The extent of the projected raster was %(e1)s, while the extent of the template raster is %(e2)s. (The extent values are ordered LEFT, BOTTOM, RIGHT, TOP.) If the extent of the template is larger than the projected raster, it may mean that the template is too large (i.e. that the predictor raster does not cover the entire extent of the template). In that case, try reducing the extent of the template. If that is not the problem, try projecting the predictor raster yourself prior to providing it to this tool, such that its coordinate system, cell size, and extent match that of the template.') % {u'raster': rasters[i], u'e1': describeTemp.Extent, u'e2': extent}))
 
                 # If this is ArcGIS 9.3, reset the SnapRaster ArcGIS
@@ -1159 +1177 @@
                     pass
 
-            # Update our dictionary to use the projected
-            # raster.
-
-            rastersForPredictors[vars[i]] = tempRaster
+        # If we need to clip (but not project), do it now. This will
+        # also take care of copying the raster, if needed.
+                
+        elif needToClip:
+
+            # Clip the raster. If this is ArcGIS 9.2, pass rasters[i]
+            # for the in_template_dataset parameter, to work around
+            # the 9.2 bug described in
+            # http://forums.esri.com/Thread.asp?c=93&f=1729&t=230897&mc=11.
+            
+            tempRaster = os.path.join(tempDir.Path, u'clipped%i' % i)
+            if GeoprocessorManager.GetArcGISMajorVersion() == 9 and GeoprocessorManager.GetArcGISMinorVersion() == 2:
+                gp.Clip_management(rasters[i], extent, tempRaster, rasters[i])
+            else:
+                gp.Clip_management(rasters[i], extent, tempRaster)
+
+            # Even though we provided an extent of an existing raster
+            # (the template raster) that has the same cell size and is
+            # aligned on the same coordinates, using values we
+            # obtained through ArcGIS APIs, the Clip tool will not
+            # necessarily create a raster that has exactly that
+            # extent. It may be one cell larger or smaller in any of
+            # the four directions. Different versions of ArcGIS seem
+            # to work differently in this respect. It is probably due
+            # to whether it uses 32-bit or 64-bit floats, and other
+            # various floating-point rounding issues.
+            #
+            # The prediction code in R requires that all predictor
+            # rasters have the same extent (really, the same number of
+            # rows and columns). If ArcGIS produced a raster that was
+            # too small or large in any dimension, try Clip again with
+            # values that will hopefully yield what we want.
+
+            describeTemp = gp.Describe(tempRaster)
+            newEdges = _CheckResultingEdges(tempRaster, describeTemp.Extent, templateLeft, templateBottom, templateRight, templateTop, cellSize)
+
+            if newEdges is not None:
+                Logger.Debug(_(u'Reclipping because the first attempt produced a raster with an extent (%(e1)s) that did not match the extent of the template raster (%(e2)s).') % {u'e1': describeTemp.Extent, u'e2': extent})
+                tempRaster = os.path.join(tempDir.Path, u'clipped%ia' % i)
+                if GeoprocessorManager.GetArcGISMajorVersion() == 9 and GeoprocessorManager.GetArcGISMinorVersion() == 2:
+                    gp.Clip_management(rasters[i], u' '.join(map(unicode, newEdges)), tempRaster, rasters[i])
+                else:
+                    gp.Clip_management(rasters[i], u' '.join(map(unicode, newEdges)), tempRaster)
+
+        # If we got here but did not need to project or clip, we
+        # needed to copy. Create a copy of the raster in ArcInfo
+        # binary grid format.
+
+        else:
+            tempRaster = os.path.join(tempDir.Path, u'copied%i' % i)
+            ArcGISRaster.CopySilent(rasters[i], tempRaster)
+
+        # If we had to project or clip, make absolutely sure that the
+        # projected or clipped raster has the expected number of rows
+        # and columns.
+
+        if needToProject or needToClip:
+            d = gp.Describe(tempRaster)
+            if d.Height != templateDescribe.Height or d.Width != templateDescribe.Width:
+                Logger.RaiseException(RuntimeError(_(u'Internal error in this tool. Please contact the MGET development team for assistance. Error details: after %(raster)s was projected and/or clipped, the resulting raster had %(cols1)i columns and %(rows1)i rows, while the template raster has %(cols2)i columns and %(rows2)i rows. This tool expected that the resulting raster would have the same number of rows and columns as the template raster. You can work around this problem by projecting and/or clipping the raster yourself, so it has the same coordinate system, cell size, and extent as the template raster.') % {u'raster': rasters[i], u'rows1': d.Height, u'cols1': d.Width, u'rows2': templateDescribe.Height, u'cols2': templateDescribe.Width}))
+
+        # Update our dictionary to use the projected/clipped/copied
+        # raster.
+
+        rastersForPredictors[vars[i]] = tempRaster
+
+    # Pass the dictionary that maps predictor variables to rasters to
+    # R, so the R code can read it.
 
     r['rastersForPredictors'] = rastersForPredictors
+
+def _CheckResultingEdges(resultingRaster, resultingExtent, templateLeft, templateBottom, templateRight, templateTop, cellSize):
+    from GeoEco.Types import EnvelopeTypeMetadata
+
+    [resultingLeft, resultingBottom, resultingRight, resultingTop] = EnvelopeTypeMetadata.ParseCoordinatesFromString(resultingExtent)
+    needToTryAgain = False
+    newEdges = [None, None, None, None]
+    
+    if resultingLeft < templateLeft - cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[0] = templateLeft + cellSize
+    elif resultingLeft > templateLeft + cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[0] = templateLeft - cellSize
+    else:
+        newEdges[0] = templateLeft
+    
+    if resultingBottom < templateBottom - cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[1] = templateBottom + cellSize
+    elif resultingBottom > templateBottom + cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[1] = templateBottom - cellSize
+    else:
+        newEdges[1] = templateBottom
+    
+    if resultingRight < templateRight - cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[2] = templateRight + cellSize
+    elif resultingRight > templateRight + cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[2] = templateRight - cellSize
+    else:
+        newEdges[2] = templateRight
+    
+    if resultingTop < templateTop - cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[3] = templateTop + cellSize
+    elif resultingTop > templateTop + cellSize * 0.01:
+        needToTryAgain = True
+        newEdges[3] = templateTop - cellSize
+    else:
+        newEdges[3] = templateTop
+
+    if needToTryAgain:
+        return newEdges
+    return None
 
 
@@ -1199 +1328 @@
     arcGISDisplayName=_(u'Fit GLM'),
     arcGISToolCategory=_(u'Statistics\\Model Data\\Generalized Linear Models'),
-    dependencies=[ArcGISDependency(9, 1), RDependency(2, 5, 0)])
+    dependencies=[ArcGISDependency(9, 1), RDependency(2, 6, 0)])
 
 AddArgumentMetadata(GLM.FitToArcGISTable, u'cls',
@@ -1623 +1752 @@
     arcGISDisplayName=_(u'Predict GLM From Rasters'),
     arcGISToolCategory=_(u'Statistics\\Model Data\\Generalized Linear Models'),
-    dependencies=[ArcGISDependency(9, 2), RDependency(2, 5, 0), RPackageDependency(u'rgdal')])
+    dependencies=[ArcGISDependency(9, 2), RDependency(2, 6, 0), RPackageDependency(u'rgdal')])
 
 CopyArgumentMetadata(GLM.FitToArcGISTable, u'cls', GLM.PredictFromArcGISRasters, u'cls')
@@ -1719 +1848 @@
 
 AddArgumentMetadata(GLM.PredictFromArcGISRasters, u'templateRaster',
-    typeMetadata=ArcGISRasterTypeMetadata(mustExist=True, mustBeDifferentThanArguments=[u'inputModelFile', u'outputResponseRaster'], canBeNone=True),
+    typeMetadata=ArcGISRasterLayerTypeMetadata(mustExist=True, mustBeDifferentThanArguments=[u'inputModelFile', u'outputResponseRaster'], canBeNone=True),
     description=_(
 u"""Template raster that defines the coordinate system, extent, and
@@ -1877 +2006 @@
     arcGISDisplayName=_(u'Fit GAM'),
     arcGISToolCategory=_(u'Statistics\\Model Data\\Generalized Additive Models'),
-    dependencies=[ArcGISDependency(9, 1), RDependency(2, 5, 0)])
+    dependencies=[ArcGISDependency(9, 1), RDependency(2, 6, 0)])
 
 AddArgumentMetadata(GAM.FitToArcGISTable, u'cls',
@@ -2059 +2188 @@
     arcGISDisplayName=_(u'Predict GAM From Rasters'),
     arcGISToolCategory=_(u'Statistics\\Model Data\\Generalized Additive Models'),
-    dependencies=[ArcGISDependency(9, 2), RDependency(2, 5, 0), RPackageDependency(u'rgdal')])
+    dependencies=[ArcGISDependency(9, 2), RDependency(2, 6, 0), RPackageDependency(u'rgdal')])
 
 CopyArgumentMetadata(GAM.FitToArcGISTable, u'cls', GAM.PredictFromArcGISRasters, u'cls')
@@ -2091 +2220 @@
     arcGISDisplayName=_(u'Predict Bayesian Probabilities for GAM From Rasters'),
     arcGISToolCategory=_(u'Statistics\\Model Data\\Generalized Additive Models'),
-    dependencies=[ArcGISDependency(9, 2), RDependency(2, 5, 0), RPackageDependency(u'mgcv'), RPackageDependency(u'MASS'), RPackageDependency(u'rgdal')])
+    dependencies=[ArcGISDependency(9, 2), RDependency(2, 6, 0), RPackageDependency(u'mgcv'), RPackageDependency(u'MASS'), RPackageDependency(u'rgdal')])
 
 CopyArgumentMetadata(GAM.FitToArcGISTable, u'cls', GAM.BayesPredictFromArcGISRasters, u'cls')
@@ -2209 +2338 @@
     arcGISDisplayName=_(u'Plot Performance of Binary Classification Model'),
     arcGISToolCategory=_(u'Statistics\\Model Data\\Evaluate Model Performance'),
-    dependencies=[RDependency(2, 5, 0), RPackageDependency(u'ROCR')])
+    dependencies=[RDependency(2, 6, 0), RPackageDependency(u'ROCR')])
 
 AddArgumentMetadata(ModelEvaluation.PlotPerformanceOfBinaryClassificationModel, u'cls',
@@ -2552 +2681 @@
     arcGISDisplayName=_(u'Plot ROC of Binary Classification Model'),
     arcGISToolCategory=_(u'Statistics\\Model Data\\Evaluate Model Performance'),
-    dependencies=[RDependency(2, 5, 0), RPackageDependency(u'ROCR')])
+    dependencies=[RDependency(2, 6, 0), RPackageDependency(u'ROCR')])
 
 CopyArgumentMetadata(ModelEvaluation.PlotPerformanceOfBinaryClassificationModel, u'cls', ModelEvaluation.PlotROCOfBinaryClassificationModel, u'cls')

MGET/Branches/Jason/PythonPackage/src/GeoEco/Statistics/PredictLMForArcGISRasters.r

@@ -72 +72 @@
                 tryCatch(
                 {
-                    if (any(rasterInfo2@grid@cells.dim != rasterInfo2@grid@cells.dim))
+                    if (any(rasterInfo2@grid@cells.dim != rasterInfo@grid@cells.dim))
                         stop("All of the predictor rasters must have the same number of rows and columns.", call.=FALSE)
-
-                    if (any(rasterInfo2@bbox != rasterInfo2@bbox))
-                        stop("All of the predictor rasters must have the same geographic extent and cell size", call.=FALSE)
                 }, finally=close(rasterInfo2))
             }