Geocoding Thousands of Fiscal Records: Methodological Approach for a Study on Urban Retail Trade in the Belle Époque

In August 1902 the British newspaper The Times inserted an article announcing “The passing of the grocer” as a result of a crisis in the small retail trade that apparently had bankrupted more than 900 stores (Winstanley, 1983). Across the Atlantic, Canada’s shopkeepers faced a crisis equally significant, with overabundance of shops and a lot of bankruptcies (Monod, 1996). The same happened in continental European cities like Paris or Milan, in the final decades of the nineteenth century (Nord, 1986; Morris, 1993). Much of what happened this time was the result of an demographic growth in the cities in the nineteenth century that almost everywhere would have important consequences in the reorganization of urban space, leading to changes in their economic and social geography. Lisbon was no exception and the city’s retail trade, after a phase of growth in the 1880s, faced a crisis in the last ten years of the century (Alves 2012). This occurred while the city was transformed, witnessing an expansion into new urban areas to accommodate an increasing volume of inhabitants.

Through the analysis of the geographical distribution of Lisbon’s retail trade, between 1890 and 1910, it is possible to verify the impact of this crisis on the way small businessmen apprehended the urban space, as well as the opportunities and risks that those changes could represent to them. The study is based on a very detailed information about the localization and characteristics of every single shop in the city streets, gathered from fiscal sources in the municipal archives (around sixteenth thousand records for each of the three years, 1890, 1900 and 1910) and analyzed through the use of spatial statistical analysis tools. This volume of information and the potential introduced by a Digital Humanities / Spatial Humanities approach, brings in methodological challenges regarding the digitization and geocoding process of several thousands of records, as well as opens new research possibilities and new research questions, namely: about the role of women in the retail trade business, since previous work based on smaller sets of data analyzed with traditional qualitative methods had misrepresented or even ignored gender issues in the retail business (Alves, 2012); or about the influence of the dwelling rents in determining changes in the social space of a city in the Belle Époque, only possible by the digital analysis of a very large, temporal and geographical encompassing amount of data for all the city.

In terms of the methodology two were the challenges we faced in two distinct stages of research, data collection and its georeferencing. The first challenge relates to the difficulty in scanning a volume of data of this nature, only available on archive, with no available funding to carry out a project of mass digitization of the original tax records. This was associated with the fact that all sources are handwritten, drawn up by several employees, with equally different handwritings, which would make it impossible, given the current state of handwritten character recognition technology (Beatty, 2010; Brumfield, 2014), for an automatic or even semiautomatic data treatment. The solution followed a close approach to crowdsourcing projects (Causer et al., 2012), using shared databases and collaborative work, a method already developed with excelent results in previous studies (Alves and Queiroz, 2013; Alves and Queiroz, 2015). The second challenge went through georeference about fifty thousand addresses obtained in this documentation, using essentially the computer's processing power. It is recognized that there are advantages and disadvantages in using several geocoding methods (Zandbergen, 2008). Its application to Lisbon is made difficult due to the fact that the city have gone through a deep urban morphology transformation and experienced significant changes in its street names throughout the twentieth century (Alves, 2005; Oliveira and Pinho, 2006). Nevertheless, it remains one of the best methods for automatic assignment of geographic coordinates. The challenge was to think of a process that could overcome the difficulties listed, without having to go through the full reconstitution of the urban network. Not least because the available sources do not made possible the recreation of all the buildings and their functional classification in a GIS environment, as was achieved in other projects (Dunae et al., 2013). The option went through reconstruction of the existing streets network of the time, based on geo-referenced digital cartography. Thus an addresses’ database was created, with slight adaptations on the geocoding algorithm of the GIS platform used, allowed for a success rate of around 90%.

As for georeferencing, the first and greatest of all the problems is related to the urban changes that the city has undergone over the last 120 years. On the one hand, strong population growth led to the expansion or changes in the corresponding urban area, even If we take into account just the three years analyzed. The city of 1890 is very different from the one in 1910, in the layout of the streets, in the expansion into new areas, construction or renovation of its buildings. On the other hand, Lisbon had the particularity to overcome, between 1890 and the present, four very different political regimes that have left a peculiar and deeply transformative mark in the city's toponomy. Even if the urban area was stable, only the change of street names over more than a century of profound political changes, would pose great challenges to an automated geocoding process.

For this there was first the need to rebuild the streets network of the time, trying to recreate the map of Lisbon of the Belle Epoque. This map was fixed for 1890 and then it was possible to apply the normal geocoding techniques, adapting either the collected data source, or the software algorithm used, in order to overcome difficulties so small, but so significant in the final results, such as the fact that the software was incapable to deal with some Portuguese names and characters or to recognize certain types of streets that were used at the time but ignored or underused today. Or the fact that the names of some streets are identical or very similar in different areas of the city. This is an iterative process of trial and error so as to refine the best model data that maximizes the results obtained.

However, there was still the issue of street names changing over time. For the period in question this problem is not very complicated, since only at the end of 1910 took place the first regime change, the Republican Revolution, with the consequent wave of place names changed. But there was almost annually, specific changes in street names. These changes were incorporated in the database, maintaining the address structure already georeferenced for 1890, allowing to incorporate the data for the same streets that appeared in the sources with different names.

These two ways to overcome the problems with georeferencing were only possible due to the existence of sources, either cartography or streets itineraries, which allowed to go adapting the original map so as to incorporate, with a similar success rate, the 1900 and 1910 data.

As for the quality or accuracy of georeferencing, it is obvious that the ideal would be to have the possibility to rebuild not only the network of city streets, but also the actual location of the various housing/trade blocs. Unfortunately no sources for this level of detail are available. In the original shapefile map, which represented the actual city streets, in 2012, the streets were targeted with the actually existing building numbers in each block. Since we have no way of saying, at this stage of the research, how many buildings existed on every block in 1890, it was decided to distribute all points along the entire length of the street, according to the geocoding algorithm. Obviously, this option can cause some significant deviation in less consolidated urban areas of the city, but it is also true that the areas where the overwhelming majority of small businesses traditionally where implemented account for already established streets. So that problem here is much lower, and the geocoding accuracy is much higher.

I did not use crowdsourcing primarily due to the available time for the collection of data and to poor accessibility to the original sources. Seems odd to state this because apparently these are precisely justifications for using crowdsourcing. However, it is known, also through the literature already cited in the abstract, that the use of crowdsourcing can be a lengthy process and not always easy to check on the quality of final results. Furthermore, it requires easier access to sources through digital copies because not all potential participants have availability or even competence for archive work. In our case, the sources correspond to thick volumes of bound sheets, deposited in a municipal archive without major conditions for local consultation and without the financial ability to carry out its full or partial scan. There was also the problem of the quality of the data, given that we are talking about handwritten information sometimes difficult to read. In this sense, the use of a collaborative work, through volunteer history students with availability and sensitivity to archival work was decisive. This data collection was made via a PostgreSQL database, shared with all students through ODBC, so that all data collected by one of the students become available for subsequent use by others. This process, already tested in other project with very good results (Alves and Queiroz, 2013; Alves and Queiroz, 2015) , saved up a lot of time in data collection and redundancies were avoided, because a new address o r a new commercial activity detected by one of the students in the sources once registered in the database, was automatically available and could be used/selected by all the others. Even if the original record contained an error, since it was registered only once in the database, because the data model prevented duplicates, the validation task was also facilitated.

The reconstruction of the 1890 street map, base of all subsequent work, was made through a current map in shapefile format, provided by the Municipality. This map was superimposed on a digital copy of an old city map properly georeferenced. The mere overlap of the two maps identified a broad set of streets that did not exist at the time (eliminated from the shapefile) and others that changed their layout (fixed in the shapefile). Using streets itineraries, namely one for 1890, very complete, mentioning the name, location and building numbers of all city’s streets, it was possible to correct the shapefile street names that had gone through changes over these 120 years. The link between this corrected shapefile and the shops’ addresses collected in the shared database allowed for geocoding, with minor adjustments to the software algorithm.

Overcoming these issues, it was possible to get a far more dynamic source, more accessible and manageable, able to respond quickly to old research questions or to introduce new perspectives about Lisbon and its retail trade in the Belle Époque. Just to mention one possibility, in the Lisbon municipal archive there are many available data that could be crossed with the information collected on shops and shopkeepers. In particular, it has data on electoral census and elections, with voters lists organized by parishes and addresses, for several years of the nineteenth and twentieth century and this data can also be mapped and analyzed using the methodology now developed. The same can happen with data on primary education, for example, because this fund has information on the addresses of the students also. Given that the addresses of the shops are being collected in a separate table and that it allows to be connected to either the GIS or to other tables with different data, whether on small businesses, on elections or on education, the use of this methodology will allow for results in other projects or to add new information relevant to the history of the city without the need to start everything from scratch.

In this sense, while recognizing the limitations of GIS (Boonstra, 2009; Bodenhamer et al., 2010; Silveira, 2014), it must be highlighted the capabilities of its spatial analysis, processing and data visualization tools, that has been particular useful in the context of urban history (Hillier, 2010; DeBats and Gregory, 2011). In recent years, the use of GIS for the study of cities and their retail trade (Beascoechea Gangoiti, 2003; Mirás Araujo, 2008; Bassols and Oyón Bañales, 2010; Novak and Gilliland, 2011; Ünlü, 2012) has enhanced the theoretical framework and the possibility of international comparative studies, which represented a definitive stimulus to the application of these methodologies to this case study. To this matter it is useful to mention that a session comparing four case studies for Iberian port cities will occur in the ESSHC in Valencia, this year, putting together comparative analysis about Barcelona, Bilbao, Coruña and Lisbon, in the first decades of the twentieth century (https://esshc.socialhistory.org/esshc-user/programme?day=55&time=145&session=3003).