Machine Learning and Data Mining

Analyses of large datasets (i.e., big data) are benefitting from advances in data mining, artificial intelligence, and machine learning. These techniques provide sophisticated ways to explore millions or billions of observations, including through learning ability, deduction ability, perception ability, and natural language ability. In other words, these techniques are attempting to imitate human intelligence. In particular, machine learning (ML) is a subfield of artificial intelligence and a way for computers to better recognize patterns through data analysis and self-study process. With these advances, computers can make decisions based on information that has not been provided. Similar to the human learning process, computers are able to generate and accumulate new information and inference to be used in iterative learning processes, thus improving their capabilities.

For example, suppose you want to distinguish whether a new article in JAPA (Journal of the American Planning Association) is about public transportation or not. The examples that the system uses to learn are called the training set. Each training example is called a training instance (or sample). In this case, task T is to distinguish whether an article is about public transportation or not, experience E is the training data, and performance measure P is the probability of accurate categorization, which is called accuracy. In other words, it is a measure of how well the article was classified.

Use of Machine Learning in Urban Planning

In the previous example, if we want to find articles on public transportation, we have to search for a few words such as transit, subway, or bus in the titles or bodies of previous publications. However, as time passes, new words will emerge relating to

public transportation such as mode split, transit equity, and transit-oriented development. It becomes more difficult to classify them accurately using traditional approaches, as classification rules become more complex. Machine learning analysis automatically recognizes frequent occurrence of these terms in an article the user has designated as about public transportation, and the machine can classify these as public transportation articles.

Through a machine learning algorithm, humans can also learn. For example, if the algorithm has been trained sufficiently to classify' public transportation articles, a human can identify' the best combination of words to correctly predict this. This is called data mining.

As an example of data mining, suppose we analyze words that have appeared in urban planning journals from the past to the present. First, we will have to instruct the machine learning algorithm to remove unnecessary words such as the, a, and this. Also, similar words such as public transportation and transit, impact and effect, and environment and environmental should be learned as the same words. Then, check the appearance and disappearance of the words. Through this process, we can identify' words that have appeared in the past, have disappeared, and have appeared recently. In addition, correlation and hierarchy of planning words can be grasped. This allows us to understand the trends in changing patterns of concepts in urban planning (see Planning Example later in the chapter).

Types of Machine Learning

Machine learning techniques can be classified into three basic categories: (1) whether the computer is trained under a human supervisor (supervised, unsupervised, semi-supervised, and reinforcement learning), (2) whether the computer is trained incrementally in real time (online learning, batch learning), or (3) whether the computer compares the data points you have with the new data points, to create a predictive model through finding a pattern in a training dataset (instance-based learning, model-based learning).

First, supervised learning is performed by injecting the algorithm with the desired answer, or label, into the training data. Classification and regression are typical supervised learning methods. Other examples of supervised learning include K-nearest neighbors, linear regression, logistic regression, Support Vector Machines (SVM), decision tree, random forest, and neural networks. These methods are used to recognize similarities across multiple dimensions to better detect patterns and, therefore, prediction.

Unsupervised learning does not have preassigned labels through training activities. In these cases, algorithms rely on inferential techniques to identify’ patterns. Examples of unsupervised learning include clustering (K-means, Hierarchical Cluster Analysis (HCA), Expectation Maximization), visualization and dimensionality reduction (Principal Component Analysis (PCA), Kernel PCA, Locally-Linear Embedding (LLE), t-distributed Stochastic Neighbor Embedding (t-SNE)), and association rule learning (Apriori, Eclat). These methods are especially useful for extremely large datasets that may be too costly to more directly train or label observations.

Semi-supervised learning is mainly used for data with only a few labels. In this case, this process is conducted by a combination of supervised and unsupervised learning.

Reinforcement learning applies when an agent performs an action through observing the environment. As a result, the agent repeats the process of receiving a reward or a penalty. This allows the agent to learn the policy to get the most compensation. DeepMind’s AlphaGo program is one of the well-known examples of reinforcement learning.

Second, offline learning is not a gradual learning process but a method that applies to the system after being trained through all the data they have. On the other hand, online learning is a method of injecting data in mini-batch units into a program and training the system and enabling learning as soon as data arrives.

Third, instance-based learning is a method that a system learns by remembering cases. If we perform classification, this process classifies by measuring the similarity of existing ones. For example, if you designate a paper about public transportation, and there are many similar words with the previous public transportation paper in new papers, the new paper is classified as a public transportation paper. Model-based learning is a method of selecting models such as linear regression and logistic regression and creates a model and uses it for predicting the value.

In addition, cautionary points in machine learning process are as follows: poor quality data, insufficient quantity of data, irrelevant features, nonrepresentative training data, and overfitting (underfitting) of the training data. Also, to understand how well the model is generalized to the sample, it is essential to test generalization error by actually applying it to the new samples. In general, the generalization error is tested by a training set and test set. In order to prove this, cross-validation is conducted to efficiently use training data without setting the separate validation set.

Planning Example

If you are a planning scholar or student, you probably have, at least once in your career, felt helpless when faced with an ever-growing pile of literature on your desk. With the advancement of technology', Assistant Professor Kerry Fang at Florida State University and Professor Reid Ewing of the University of Utah have a solution for you, and for the field of urban planning to track its progressive footsteps. In a recent paper "Tracking Our Footsteps: Thirty Years of Publication in JAPA,JPER, andJPL” (available from the authors), they have adopted a data/text mining technique to let the computer efficiently read through all the publications since 1990 in the Journal of the American Planning Association, Journal of Planning Education and Research, and Journal of Planning Literature. The final text file includes 1,463 research articles and 13 million words, too much for human reading.

Using VOSviewer, a visualization software package with a built-in text-mining technique, and a static topic model in R statistical package to conduct a complete topic analysis, they identified 14 major research themes in these publications. They are, by order of importance, planning process, planning method, plan implementation, community planning, public finance, economic development, sustainability, neighborhood planning, land use, growth management, urban design, housing, planning education, and transportation. Among these themes, some overlap with each other, i.e., frequently occur in the same article and/or share vocabulary, such as planning process and growth management, community and land use, and public finance and economic development. Others are isolated, such as planning method and planning education.

They also tracked the dynamics of research themes over time. Some themes have always remained important, such as planning process, planning method, and land use/growth management. Others were once prominent, but lost their status over time. Examples are planning theory and planning education. Planning ethics, food, and historic preservation surfaced in the most recent five years as emerging themes in planning publications.

Their paper demonstrates a text mining method to effectively grasp a large number of publications and track our footsteps in the study of urban planning. Their paper can be easily replicated for other journals, and updated every decade in the future to help track research progress, and identify research gaps, emerging trends and future directions.

Summary

This chapter briefly discussed several basic topics related to urban planning data and analysis. In many cases, these topics are not unique to planning but are relevant across many applications of data science. While analysis methods and techniques often receive the most attention, it is important to understand the data being used, why they were collected, how they were collected, and when they were collected relative to the task at hand, topics to be explored, and questions to be answered.

Note

1. wiMv.emc.com/leaclership/digital-universe/2014iview/executive-summary.htm

Works Cited

Barkham, R., Bokhan, S., & Saiz, A. (2018). Urban big data: City management and real estate markets. GovLab Digest, New York.

Batty, M. (2013). Big data, smart cities and city planning. Dialogues in Human Geography, 3(3), 274-279.

Dandekar, H. C. (Ed.). (2003). The planner’s use of information. Chicago, IL, USA: American Planning Association Planners Press.

Geron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: Concepts, tools, and techniques to build intelligent systems. Sebastopol, CA, USA: O'Reilly Media, Inc.

Kitchin, R. (2013). Big data and human geography: Opportunities, challenges and risks. Dialogues in Human Geography, 3(3), 262—267.

Ma, R., Lam, P. T. L, & Leung, C. K. (2018). Big data in urban planning practices: Shaping our cities with data. In K. Chau, I. Chan, W. Lu, & C. Webster (Eds.), Proceedings of the 21st international sy mposium on advancement of construction management and real estate. Singapore: Springer.

Mitchell, T. M. (1997). Does machine learning really work? Al Magazine, 18(3), IL

Samuel, A. J. (1959). U.S. patent no. 2,904,229. Washington, DC: U.S. Patent and Trademark Office.

Schroeck, M., Shockley, R., Smart, J., Romero-Morales, D., & Tufano, P. (2012). Analytics: The real-world use of big data. IBM Institute for Business Value report, 1-20.

Tauberer, J. (2014). Open government data: The booh. Lulu. Morrisville, NC, USA. Retrieved March 03, 2019, from https://opengovdata.io/

Thakuriah, P., Tilahun, N., & Zellner, M. (2017). Big data and urban Informatics: innovations and challenges to urban planning and knowledge discover}’. In: Thakuriah, P, Tilahun, N. and Zellner, M. (eds.) Seeing Cities Through Big Data: Research Methods and Applications in Urban Informatics. Springer: New York, pp. 11-45. ISBN 9783319409009 (doi: 10.1007/978-3-319-40902-3_2).

Wang, X., & Hofe, R. (2008). Research methods in urban and regional planning. Co-published by Tsinghua University Press, Beijing, China and Springer-Verlag GmbH Berlin Heidelberg, Germany.