Author:Wanchen Xu
Project Github Address: https://github.com/xumou1/G5990M_Assignment2
Streamlit Share Address: https://xumou1-g5990m-assignment2-assignment2-p4cr3f.streamlit.app/
Two different scenarios exist in Assignment 2. One is that a company producing stone has provided three essential indicators that influence the location of its plant in the UK and wants to obtain software that can manually adjust the weighting parameters. A certain amount of code was required to accomplish this goal. Secondly, an extreme sports holiday company wanted to find suitable locations for their activities and therefore needed you to find a suitable location by calculating the slope.
In this scenario, we needed to complete the tasks given in the scenario, and in this case, we chose Python as the primary language according to the course requirements.Here's a brief introduction to python and streamlit
Python is a high-level, interpreted, and general-purpose programming language created by Guido van Rossum and first released in 1991. It has a simple and easy-to-read syntax, making it a popular choice for beginners and experienced programmers. Python is known for its versatility. It can be used for various applications, including web development, data analysis, artificial intelligence, machine learning, scientific computing, automation, and more.
Python has an extensive standard library, which provides a wealth of built-in modules and functions to handle everyday tasks. Additionally, Python has a vast ecosystem of third-party packages available through the Python Package Index (PyPI), further extending its capabilities. Link to Python
Streamlit is a Python library for building interactive web applications for data science and machine learning. It was created in 2018 by Adrien Treuille, Amanda Kelly and Thiago Teixeira.Streamlit aims to enable data scientists and developers to quickly and easily create engaging data visualization applications without spending much time writing front-end code.
Streamlit provides a clean, easy-to-use API that lets us focus on working with data and creating visualizations without worrying about complex UI logic. It integrates with standard Python data science libraries such as NumPy, Pandas, Matplotlib and Plotly. In addition, Streamlit supports hot reloading, which means that when we update our code, the application responds and updates in real-time, making iterative development much more accessible. Link to Streamlit
In this section we will focus on analysing the project requirements in both scenarios and combine them with the requirements outlined in Assignment 2 to arrive at a more complete and mature project requirements contentMLife is short, use Python. --Guido van Rossum
First of all we can see in the pdf of the brief description of the project that there are four basic requirements for this project.
1. Reads the raster data and displays them.
2. Multiplies each raster by a factor, adds the weighted rasters together and rescales the resulting raster to have values in the range [0, 255].
3. Displays the result raster and writes this to a file.
4. Provides a GUI that allows the user to choose the weights by means of ‘sliders’ and that displays the result.
This is a program that requires the development of a graphical user interface (GUI) whose primary function is to read one or more raster data and perform a weighted multiplication operation, add the results and rescale them to values in the range [0, 255] and display the resulting raster data in the GUI. The program also provides sliders for users to select weights and save the results to a file. The requirement is primarily for geospatial data analysis. It is intended to provide the user with a simple and practical tool to adjust the weights and visualise the results for spatial analysis using the sliders.
We can see that in the pdf of the brief project, the primary four requirements for this project are briefly the four steps of the experiment and the visualisation of reading data, processing data, producing results and exporting results. The requirement is that the final results be regularised to between 0 and 255, as well as allowing the user to control the processing of the data steps (and adjusting the weights), again based on our requirements. We added the ability to query and edit the original data and download the updated data. We found that it was not easy to find a range of problems with the data when querying the table for complete UK data, so we added the ability to image the data. The requirements for this project are similar to those of the previous one, and the documentation briefly mentions four requirements for it.
1. Reads the raster data and displays it.
2. Calculates the maximum slope for each row and column (pixel) of the raster by considering all the slopes between the pixel and the 8 nearest pixels. (Think about what to do at the map edges.)
3. Saves the maximum slopes to a file in the same format as the provided raster data.
4. Provides a GUI that allows the user to choose the raster data file to load and that displays the result and on top of this the most extreme locations in terms of maximum slope.
Both requirements require the development of a GUI program and involve the reading, processing and display of the raster data and the saving of the results to a file. However, their primary function is different, as the main function of this project requirement is to calculate the maximum slope of the raster data and display the result in the GUI. In contrast, the primary function of the first requirement is to perform a weighted multiplication operation on the raster data and add the results, then rescale the result and display it in the GUI. In addition, the first requirement requires the provision of a slider for the user to select the weights. The focus of this project requirement was to calculate the maximum slope of the raster data and display the result in the GUI. This required complex calculations and processing of the raster data, including consideration of map edges and the use of progress bars to track the progress of the calculations. The main objective of this requirement is to provide a practical tool for geospatial analysis that can help users better understand terrain features and landscape structure. This section describes the software functionality and the functional implementation logic of the project requirements mentioned above.
The function of this code is to add a file upload control to the Streamlit application that allows the user to select a text file and read that file as a Pandas DataFrame. This particular file upload control is used to select the geology data file. If the user uploads a file, it is read as a Pandas DataFrame, stored in the variable geology and can then be used in subsequent code. It follows the steps below:
1. Call the st.file_uploader() method to create a file upload control, displaying the text "Select geology file(txt)" and restricting the file type to "txt".
2. Check if the uploaded file is empty, and if not, read the contents of the file using Pandas' read_csv() method, specifying the file separator as a comma (sep=',') and no header line (header=None).
3. Store the read data in the variable geology for subsequent use.
As shown in code fragment 1 below
# CODE FRAGMENTS 1
# Upload the data of geology, transport, population
uploaded_file = st.file_uploader("Select geology file(txt)", type="txt")
if uploaded_file is not None:
geology = pd.read_csv(uploaded_file, sep=',', header=None)# CODE FRAGMENTS 2
new_data = st.experimental_data_editor(checking_data)
txt = convert_df(new_data)
st.download_button(
label="Download data as TXT",
data=txt,
file_name='new_' + option + '.txt',
mime='text/plain',
)# CODE FRAGMENTS 3
plt.axis('off')
plt.imshow(checking_data)
title_font = {'fontname': 'Times New Roman', 'fontsize': 15, 'color': 'black'}
plt.title(option.upper() + ' DATA', **title_font, pad=10)
plt.colorbar()
st.pyplot()
plt.close()
image_buffer = io.BytesIO()
plt.axis('off')
plt.imshow(checking_data)
title_font = {'fontname': 'Times New Roman', 'fontsize': 15, 'color': 'black'}
plt.title(option.upper() + ' DATA', **title_font, pad = 10)
plt.colorbar()
plt.savefig(image_buffer, format='PNG', dpi=600, bbox_inches='tight')
plt.close()
image_buffer.seek(0)
st.download_button(
label="Download image",
data=image_buffer,
file_name=option + ".png",
mime="image/png",
)# CODE FRAGMENTS 4
geology_weight = st.sidebar.slider("Geology Weight", 0.0, 100.0, 30.0)
transport_weight = st.sidebar.slider("Transport Weight", 0.0, 100.0, 30.0)
population_weight = st.sidebar.slider("Population Weight", 0.0, 100.0, 30.0)# CODE FRAGMENTS 5
def Normalize(Dataframe):
'''
Test for Normalize function.
>>> df = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6], 'c': [7, 8, 9]})
>>> Normalize(df)
a b c
0 0.0 31.875 63.75
1 95.625 127.5 159.375
2 191.25 223.125 255.0
'''
return (Dataframe - Dataframe.min().min()) * (255 - 0) / (Dataframe.max().max() - Dataframe.min().min())The code uses the D8 algorithm, which calculates the slope of each pixel of the DEM and returns an array of the same size as the DEM: Extend the DEM by 1 unit in both rows and columns, using "edge extrapolation", where the elevation of a grid point on the edge is the elevation of its neighbouring interior point; Iterate through the interior of the DEM one by one and calculate the difference in height between the eight surrounding points and the point as dx (for distance in the x-direction) and dy (for distance in the y-direction) respectively; The dx and dy are used to calculate the slope values using the Pythagorean theorem, and the resultant array is returned with the same size as the original DEM array. The D8 algorithm gets its name from the fact that it uses a 3 x 3 neighbourhood, as shown in the following figure:
Where p is the currently calculated pixel point and 1-8 are the eight adjacent pixel points, numbered sequentially. In the D8 algorithm, each pixel point can only flow out in one of the eight adjacent directions, i.e. to the lowest adjacent pixel point. If multiple adjacent pixel points of the same height exist, one direction is chosen at random. This method is often used in slope calculations.
# CODE FRAGMENTS 1
def calculate_slope(dem, cell_size):
'''
Calculate the slope from a given DEM and cell size.
Args:
dem (ndarray): 2D array representing the digital elevation model.
cell_size (float): Size of a cell in the DEM.
Returns:
ndarray: 2D array representing the slope.
Examples:
>>> dem = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
>>> calculate_slope(dem, 1)
array([[0.41421356, 0.41421356, 0.41421356],
[0.41421356, 0.41421356, 0.41421356],
[0.41421356, 0.41421356, 0.41421356]])
'''
pad_dem = np.pad(dem, ((1, 1), (1, 1)), mode='edge')
slope = np.zeros_like(dem)
progress_bar = st.sidebar.progress(0)
frame_text = st.sidebar.empty()
for i in range(1, pad_dem.shape[0] - 1):
progress_bar.progress((i + 1) / pad_dem.shape[0])
frame_text.text("Calculate Progress %d percent" % (int(i/3)))
for j in range(1, pad_dem.shape[1] - 1):
dz_dx = ((pad_dem[i - 1, j + 1] + 2 * pad_dem[i, j + 1] + pad_dem[i + 1, j + 1]) -
(pad_dem[i - 1, j - 1] + 2 * pad_dem[i, j - 1] + pad_dem[i + 1, j - 1])) / (8 * cell_size)
dz_dy = ((pad_dem[i + 1, j - 1] + 2 * pad_dem[i + 1, j] + pad_dem[i + 1, j + 1]) -
(pad_dem[i - 1, j - 1] + 2 * pad_dem[i - 1, j] + pad_dem[i - 1, j + 1])) / (8 * cell_size)
slope[i - 1, j - 1] = np.sqrt(dz_dx ** 2 + dz_dy ** 2)
return slope# CODE FRAGMENTS 2
slope_without_nan = np.ma.masked_where(np.isnan(slope), slope)
max_slope_position = np.unravel_index(np.argmax(slope_without_nan), slope_without_nan.shape)
min_slope_position = np.unravel_index(np.argmin(slope_without_nan), slope_without_nan.shape)
plt.plot(max_slope_position[1], max_slope_position[0], 'rx', label='Max Slope', markersize = 5, markeredgewidth = 2)
plt.plot(min_slope_position[1], min_slope_position[0], 'ro', label='Min Slope', markersize = 5, markeredgewidth = 2)Before using the project, users need to deploy the project. Of course, the project has been laid out on Streamlit Cloud, which can be accessed through the link at the beginning of the README document, below we demonstrate the deployment of the project from both online and local directions.
First, we go to the Streamlit cloud website and log in via our Github account to complete the link to the repository. Here is the link to the Streamlit cloud. The steps are shown below. [Streamlit Share Link](https://share.streamlit.io/)
First we need to download the Github repository locally
git clone https://github.com/xumou1/G5990M_Assignment2.git
pip install -r requirements.txt
cd ../G5990M_Assignment2
streamlit run Assignment2.py
# if streamlit can not use directly
python -m streamlit run Assignment2.py
