The Hidden Bottleneck in GIS Is Often Data Preparation
A geographic question can sound simple long before the available data is capable of answering it.
Which communities are more than thirty minutes from a health facility? Which buildings fall inside the flood-prone area? Where should a new service point be placed? How has access changed since the previous year?
The visible analysis behind these questions may be familiar. Calculate travel times, intersect two layers, count features, compare scenarios, or classify the result. In a prepared environment, some of those operations can run in seconds.
The work that precedes them is usually less visible.
The facility list may be stored in a spreadsheet with incomplete coordinates. Several names may refer to the same place. A report may contain the most recent operating status, but only in prose. Population figures may use one set of administrative boundaries while the available geometry uses another. Road data may be current in the main city and incomplete elsewhere. Dates, units, licences, coordinate systems, and definitions may differ across Sources that appear compatible when viewed separately.
Before the analytical question can be answered, someone has to decide what each record means, whether the Sources can be combined, which gaps matter, and what level of uncertainty the intended use can tolerate.
This is why the hidden bottleneck in GIS is often not the spatial operation itself. It is the preparation required to make the operation meaningful.
The Most Visible Step Is Not Always the Most Expensive
Analysis and cartography tend to receive the most attention because they produce the visible result. A buffer appears, a route is calculated, a surface changes colour, and a pattern becomes available for interpretation. Data preparation, by contrast, often looks like a collection of preliminary chores that should be completed before the “real” work begins.
Research on data analysis has repeatedly challenged that distinction. Interviews with analysts have documented recurring difficulty around acquiring, restructuring, integrating, and validating data before analysis can proceed.[^1] Work on tidy data reaches a related conclusion from another direction: consistent structure reduces the repeated manipulation required before analytical tools can operate effectively.[^2]
Neither source measures every GIS workflow, but together they describe a familiar imbalance. The transformation that receives the least public attention can occupy a substantial part of the actual work, especially when several imperfect Sources must be made compatible before the first spatial operation can be trusted.
The imbalance becomes sharper in spatial analysis because the data must agree in more than tabular structure. Records need locations, geometries need meaning, coordinate reference systems need to be understood, and spatial relationships need to remain valid after datasets are combined. A successful join can still be geographically wrong. A valid polygon can still represent the wrong year or administrative definition. A point can have precise coordinates while referring to a facility that no longer exists.
The operation may run without error because software can confirm that the inputs are technically compatible. Whether they are appropriate for the question is a different matter.
Geographic Data Can Be Messy in Several Dimensions at Once
Ordinary data preparation already involves missing values, inconsistent categories, duplicate records, incompatible identifiers, and ambiguous dates. Geographic information adds another set of ways in which apparently reasonable data can fail.
A geometry may be missing, invalid, or more precise than the source evidence justifies. Coordinates may use an unknown or incorrectly declared reference system. Two boundary datasets may describe the same named areas but follow different revisions, scales, or surveying practices. A road may be present as a line without carrying the access restrictions needed for routing. A facility may be mapped accurately but categorised too broadly for the intended analysis. A dataset may be complete for one city and sparse in the surrounding region without making that imbalance obvious.
Some problems appear immediately when the map is drawn. Others remain hidden until several Sources are combined.
A list of schools and a set of neighbourhood boundaries may each look correct, yet the analysis can fail if the school records use informal place names while the boundaries use administrative codes. Two elevation datasets can cover the same territory while representing different resolutions and vertical datums. A report and an existing Source may describe the same assets using slightly different names, leaving the analyst to decide whether the records should be merged.
These examples reflect a larger principle: geographic preparation is not merely about making a file readable. It is about making several forms of meaning line up well enough for a particular use, while preserving enough information about dates, coordinate assumptions, provenance, and limitations for later users to understand what was done.
Clean Data Is Not the Same as Fit-for-Purpose Data
The word clean suggests that data quality is an intrinsic property. A dataset is clean or dirty, correct or incorrect, ready or unfinished.
In practice, quality depends heavily on what someone intends to do with the data.
A settlement dataset may be suitable for a national overview while being too coarse for local service planning. A road network may support visual reference without carrying the attributes required for accessibility analysis. A building layer may be sufficiently complete for estimating urban density but inappropriate for identifying every household affected by a proposed intervention. An administrative boundary can be authoritative and still be the wrong version for comparison with older statistics.
Research on spatial data quality has therefore emphasised fitness for use: whether the characteristics and limitations of a dataset are appropriate for the decision or analysis being attempted. Devillers and colleagues observed that geospatial users often struggle to access and interpret quality information, especially when heterogeneous datasets must be integrated, which increases the risk that data will be used outside the conditions it can support.[^3]
This changes the purpose of preparation.
The goal is not to remove every imperfection, because that may be impossible and unnecessary. The goal is to identify which imperfections matter, make consequential assumptions explicit, and produce a Source whose limitations are understandable in relation to its intended use.
A dataset with known gaps can be more useful than a polished dataset whose gaps are invisible. An approximate location can support regional exploration when it is represented honestly, while a falsely precise coordinate may create confidence the evidence does not justify. Preparation is therefore as much about documenting uncertainty as correcting errors.
Preparation Is an Interpretive Part of the Analysis
Many preparation tasks appear mechanical when described after the fact. Standardise the names, geocode the addresses, join the table, remove duplicates, convert the units, and repair the geometries.
Each instruction conceals a decision.
Standardising a name requires deciding which variation represents the same place and which variation represents a genuinely different feature. Geocoding requires choosing among candidates, deciding what level of precision is acceptable, and handling records that cannot be located confidently. Removing duplicates requires defining identity: two records may share a name but represent separate facilities, or use different names while referring to the same facility. Joining Sources requires deciding whether an unmatched record is an error, an absence, or evidence that the categories do not align.
Even a conversion as ordinary as turning a document into structured geographic features involves interpretation. A report may mention a village, a project area, and a service facility in the same paragraph without stating whether each should become a point, a polygon, an attribute, or contextual evidence. It may describe a location relative to another place rather than provide coordinates. The most important limitation may appear in a footnote, table heading, or date range rather than in the main record.
This interpretive character explains why preparation can become the real analytical bottleneck. The difficult part is not always performing the transformation. It is deciding what transformation preserves the meaning of the evidence.
Hadley Wickham’s work on tidy data showed the value of consistent structure: when variables, observations, and observational units are represented predictably, a broad range of analytical tools can operate with less repeated manipulation.[^2] Geographic workflows need that structural consistency, but they also need decisions about location, scale, time, lineage, and fitness for use. A table can be tidy while its geography remains unsuitable.
The Cost of Preparation Is Often Paid Repeatedly
A common GIS workflow solves the preparation problem only for the immediate task.
An analyst downloads several files, corrects a few fields, creates a local join, copies values from a report, and exports the resulting layer. The map is completed, but the decisions that made the data usable remain scattered across a project file, spreadsheet, script, temporary folder, or the analyst’s memory.
When a similar question appears later, much of the work begins again.
The Source may have changed names. The corrected identifiers may exist only in one project. Nobody may remember why certain records were excluded or which version of a boundary was used. Another analyst may repeat the same cleaning with slightly different assumptions and produce a result that cannot be compared directly with the first.
The bottleneck is therefore not only the amount of preparation required once. It is the failure to turn preparation into a reusable asset.
A prepared geographic Source should be able to outlive the map or analysis that first required it. That means preserving useful metadata, dates, provenance, licence information, known limitations, and the distinction between original evidence and derived or proposed values. It also means resisting the temptation to overwrite uncertainty until the data appears cleaner than it really is.
When the result of preparation becomes a reusable Source, later work can begin from an inspected foundation rather than an undocumented intermediate file. The preparation effort then compounds positively: one careful decision can support several Projects, Maps, and audiences.
Automation Should Reduce Repetition Without Hiding Evidence
Data preparation is an obvious target for AI because much of it is repetitive, language-heavy, and distributed across inconsistent Sources. A system can help identify likely features in a document, suggest coordinates, propose categories, match records, normalise values, or add attributes from supporting evidence.
Used well, this can move the bottleneck.
Instead of manually transcribing every candidate record, a user can review a structured proposal. Instead of researching each existing feature from the beginning, they can inspect suggested enrichment and focus attention on uncertain or consequential values. The time saved can be spent assessing the data rather than merely moving it between formats.
The danger is that automation can make interpretation look like extraction.
A system may return a place name, coordinate, category, or operating status in a clean field, even when the underlying evidence was ambiguous. The structured output can appear more authoritative than the document or inference from which it came. If the proposal is accepted silently, uncertainty is converted into apparent fact.
The appropriate model is therefore not automatic truth production. It is assisted preparation with visible evidence and review.
A useful workflow should distinguish proposals from accepted data, allow uncertain records to be corrected or rejected, preserve provenance where possible, and avoid silently replacing an existing value simply because another value appears plausible. Automation becomes valuable not when it removes the person from the workflow, but when it concentrates human attention on the decisions that require judgement.
Data Preparation in Geobble
Geobble treats preparation as part of the geographic workflow rather than as an invisible task that must happen elsewhere.
A Source can begin with an uploaded geospatial file, an external PostGIS table, or a public Source that is already available for reuse. When the necessary geography exists mainly in documents, URLs, text, or other supporting material, Extract Features can be used to produce a proposed set of geographic features and attributes. When an existing Source lacks information needed for a question, Enrich Source can propose additional values for its features.
The word propose is important.
Extraction is not treated as a promise that the system has discovered ground truth, and enrichment is not intended to overwrite uncertain facts without inspection. The user reviews the proposed features, attributes, evidence, and provenance, then accepts, corrects, or rejects the result before saving it as a Source.
Once accepted, the prepared Source can participate in the rest of the platform. It can be used in Projects, refined or combined in Studio, related to other resources, and published when its permissions and licence allow. The aim is to keep the preparation connected to the analysis and map it supports, so that the work does not disappear into a temporary preprocessing step.
This approach does not remove the difficult parts of geographic data preparation. It cannot determine automatically whether every Source is fit for a consequential decision, recover evidence that does not exist, or resolve every ambiguity in a document or place name. What it can do is make preparation easier to begin, more structured to review, and more likely to produce a reusable result.
The Bottleneck Should Become Part of the Product
There is a tendency in analytical software to design around the moment when the data is already ready.
The user arrives with a valid file, a stable schema, known identifiers, documented fields, compatible geometries, and a clear question. From that point, the product can offer analysis, visualization, and publication with impressive speed.
Real work often begins earlier.
It begins with evidence that is incomplete, distributed, inconsistently named, or only partly geographic. It begins with uncertainty about which Source should be trusted and whether several Sources describe the same world. It begins with the need to turn reports, operational records, and existing datasets into something that can support a spatial question without concealing how that transformation occurred.
When software ignores this stage, users do not stop preparing data. They perform the work elsewhere, through spreadsheets, scripts, copied values, local files, and undocumented decisions. The apparent simplicity of the product is preserved by exporting the complexity to the user.
A more complete GIS workflow should acknowledge that preparation is not an inconvenience outside the product’s concern. It is one of the places where geographic meaning is established, quality is assessed, and later conclusions become either defensible or fragile.
The spatial operation may still be the moment when the answer becomes visible. The reliability of that answer, however, was often determined much earlier, while someone was deciding what the inputs actually meant.
References
[^1]: Sean Kandel, Andreas Paepcke, Joseph M. Hellerstein, and Jeffrey Heer, “Enterprise Data Analysis and Visualization: An Interview Study,” IEEE Transactions on Visualization and Computer Graphics 18, no. 12 (2012): 2917–2926.
[^2]: Hadley Wickham, “Tidy Data,” Journal of Statistical Software 59, no. 10 (2014): 1–23.
[^3]: Rodolphe Devillers, Yvan Bédard, Robert Jeansoulin, and Bernard Moulin, “Towards Spatial Data Quality Information Analysis Tools for Experts Assessing the Fitness for Use of Spatial Data,” International Journal of Geographical Information Science 21, no. 3 (2007): 261–282.