GeoInquiries Map and Data Search

The experimental GeoInquiry search tool allows an instructional designer or advanced GeoInquiry user to search GeoInquiry maps and data for keywords (and optionally add that data to their own map).  The tool includes all Esri produced geoinquiries but also state GeoInquiries content.

For example, a search for “climate” will search the metadata (including title and description) of about 150 maps and nearly 1,000 data services while ignoring all non-GeoInquiry content in ArcGIS Online.

The database behind the tool is updated twice monthly to reflect the latest changes in map and data service metadata. Explore the tool at http://edgis.org/geoinquirysearch

screen+shot+2019-01-22+at+9.01.01+am

Harvesting ArcGIS Online Data and Maps Metadata

This short article describes a process where Python was used to harvest metadata from a list of identified ArcGIS Online maps and the maps’ data services. The data were logged to MySQL (with pymysql); a PHP web search and discovery page was created.  The process allows for keyword searching in titles and descriptions of maps and data layers.

Why harvest metadata?

This approach was used as our collection of 150 maps is housed in several AGO organizations with data services spread across even more.  These data and maps are designed for student use and vetted, making school curriculum authors interested in search and discovery of “good” data.  Essentially, we have a target population that is keenly interested in a subset of scattered data and maps.

Who might use this approach?

This article may be of interest to developers needing to create a search solution across a specific list of maps and constituent data services.

The Approach

Using the ArcGIS Python API 1.5.2 and a prebuilt list of mapIds, a script was built that iterates over the list, logging titles and descriptions for the maps.

with open('data/maps.csv') as f:
reader = csv.reader(f)
for map in reader:
result = gis.content.get(map[0])
web_map_obj = WebMap(result)
web_map_obj
web_map_obj.layers

Using the layers attribute now available for the map object, we then loop over all the layers in the map (above), also logging titles and descriptions.

We then log the data to a MySQL table (with a custom function), like:

dbwriter(
  objectType='Webmap',
  mapId=result.itemid,
  objectName=result.title,
  url='http://www.maps.arcgis.com/home/webmap/viewer.html?webmap=' + result.itemid,
  description=ssnippet
  )

Then loop over each layer and log to the table, slightly changing parameters as necessary.

You can access the script at GitHub below. The Python code, a sample CSV input file, and a sample SQL script for generating the MySQL table are included.

github  https://github.com/trbaker/mapHarvest