London House Price Prediction

Predicting house prices accurately in dynamic markets like London poses significant challenges for traditional modelling techniques due to the non-linearity of influencing factors. This project investigates the extent to which machine learning and deep learning models improve short-term prediction accuracy for freehold houses in London.

By merging HM Land Registry Price Paid Data with Energy Performance Certificate datasets, the study overcomes the feature deficiency of transactional records alone. The project follows a complete data-science lifecycle from data acquisition to model deployment.

The project follows a rigorous data-science methodology:

• Data Acquisition: Downloaded and merged HM Land Registry Price Paid Data with Energy Performance Certificate datasets under Open Government Licence v3.0.
• Data Cleaning: Handled missing values, removed outliers, filtered for London freehold properties only.
• Feature Engineering: Extracted date features (year, month, weekday, quarter), applied Label Encoding to high-cardinality categorical features (postcodes, street names).
• Data Split: Used 2015-2023 for training, reserved 2024 as hold-out set to test temporal generalization.
• Models Evaluated: Linear Regression, Random Forest, XGBoost, MLP (Neural Network), and LSTM (Deep Learning).
• Deployment: Built an interactive Gradio-based address lookup tool for end-user predictions.

Key findings from the model evaluation:

• XGBoost achieved R2 of 0.83 on the 2024 hold-out set, outperforming all other models.
• Tree-based ensembles (XGBoost, Random Forest) significantly outperformed deep learning approaches (LSTM, MLP) on this tabular dataset.
• Merging EPC data (floor area, habitable rooms, energy rating) with transaction data significantly improved predictive power over transaction data alone.

Explore the project code, notebooks, and full report: