Understanding building energy efﬁciency with administrative and emerging urban big data by deep learning in Glasgow

With buildings consuming nearly 40% of energy in developed countries, it is important to accurately estimate and understand the building energy efﬁciency in a city. A better understanding of building energy efﬁciency is beneﬁcial for reducing overall household energy use and providing guidance for future housing improvement and retroﬁt. In this research, we propose a deep learning-based multi-source data fusion framework to estimate building energy efﬁciency. We consider the traditional factors associated with the building energy efﬁciency from the Energy Performance Certiﬁcate (EPC) for 160,000 properties (30,000 buildings) in Glasgow, UK (e.g., property structural attributes and morphological attributes), as well as the Google Street View (GSV) building façade images as a complement. We compare the performance improvements between our data-fusion framework with traditional morphological attributes and image-only models. The results show that including the building façade images from GSV, the overall model accuracy increases from 79.7% to 86.8%. A further investigation and explanation of the deep learning model are conducted to understand the relationships between building features and building energy efﬁciency by using SHapley Additive exPlanations (SHAP). Our research demonstrates the potential of using multi-source data in building energy efﬁciency prediction with high accuracy and short inference time. Our paper also helps understand building energy efﬁciency at the city level to help achieve the net-zero target by 2050.


Introduction
The emergency of climate change and global warming has been recognized globally in both Paris Agreement and the Glasgow Climate Pact [1]; 153 countries have collectedly listed securing the net-zero emissions as the top missions in COP26 at Glasgow.With the building sector accounting for nearly 40% of energy consumption in developed countries [2,3], understanding buildings' energy usage and improving the energy efficiency are critical for reducing overall energy use [4].In fact, accurately predicting and understanding the building's energy efficiency is not only important for the wider objectives in global carbon emission targets, but also related to individual homeowners' decision-making and housing retrofit and improvement [5].Successful identification of households with low energy efficiency is also beneficial for eliminating fuel poverty problems [6].Numerous efforts have been devoted to predicting energy emissions [7,8], mapping energy performance [9,10], and connecting energy with real estate markets [11].
Despite its importance, multiple challenges remain for current research methods.Traditional research for energy analysis and estimation involves engineering calculation, simulation modelbased benchmarking and statistical modellings [12].Many of the current methods involve two types of data that are not readily available or difficult to obtain.Firstly, human behaviour data such as the number of occupants and heating set point temperature is often used by current research.While the energy consumption is largely affected by users' behaviour, the data is often hard to retrieve without the installation of smart metres or household surveys.Also, for rented houses, due to the high turnover of the tenants, it is difficult to have a frequent survey for the behavioural patterns.Secondly, these research often take the energy consumption related indicators into account, such as CO2 emissions, walls' solar absorptance, etc [13].The difficulty of obtaining such data and extremely high correlation between the data and the objectives make it not suitable for more extensive energy efficiency prediction research.
In this study, we present a more precise, and scalable framework for estimating building energy efficiency ratings at the city scale by using new forms of urban big data and deep learning framework.The framework is able to make classification of properties' energy efficiency ratings with building morphology description and street-level building image data.Morphological attributes have been widely used in the prediction of building energy consumption and efficiency [14].It provides information about properties' size, material, structural features and possibly implies how the property is used.The street-level building image data captures the façade of buildings and also reflects energyrelated information of the property.Architectural elements within the images, such as windows, doors and balconies are related to the style, age and structure of the buildings.What's more, it has been studied that street view images are able to reveal the relationships between built environments and socioeconomic environments [15,16].With a combination of building morphology description and street-level building images, this paper aims at achieving a comprehensive understanding of building energy efficiency.
The contribution of this paper is twofold: first, we design a scalable multi-source data fusion deep learning framework to predict building energy efficiency ratings from both building morphology attributes and street-level imagery.The framework is able to perform property-level estimation based on publicly available datasets.The openly available data, high accuracy and fine scale of the methods ensures the framework to be beneficial for realworld application and can be extended to other study areas.The incorporation of image data within the framework further improves prediction accuracy.Secondly, with the designed framework, we are able to understand the influential factors for building energy efficiency through explainable AI techniques.The framework is also able to explain how different building features contribute to the building energy efficiency estimation.It also helps the homeowners and policy maker to estimate the energy efficiency before the renovation starts.The research results are valuable in providing suggestions for the facilitation and execution of emission-reduction policies.Given the wide availability of data involved, the framework is also beneficial for broader regions rather than the presented site.

Methods for predicting building energy efficiency
In the previous research, engineering calculation, simulationbased benchmarking, data-driven statistical modellings, and artificial intelligence methods have been widely used in building energy analysis, estimation, and benchmarking [12].Engineering methodologies use physical laws to assess building energy and can achieve extremely high accuracy, but they rely upon system complex details, including mathematics and building dynamics, as well as all building components, which is not conveniently available to the public in a large area; simulation-based benchmarking includes software and computer models that have complex details and can be used for a variety of applications, but it can be very costly and time-consuming when a large number of solutions need to be defined [12].Current development of computational methods and data make it possible to use data-driven statistical models that are more efficient compared to engineering and simulation-based methods [12].Compared with other statistical models (multiple linear regression, support vector machine, decision trees, etc.), artificial neural networks has been favoured by researchers due to their reliable predictions and the advantages of overcoming the nonlinearity between the input and output of energy-related data [17,18].
Existing research mainly use the ANN-based methods to understand the building energy usage and demand [17].[19] combined ANN with the statistical method to quantify the impact of driving factors on building energy use and found that the heating/cooling degree days, the building area, the room number, and the window number are most related to the energy end-use per capita.[14] used Levenberg-Marquardt optimization algorithm to update the weights of the hidden neurons considering its high speed, and got a higher prediction accuracy rate of heat demand indicatorabout 95% of entries fall within ± 3 confidence intervals.[20] developed ANN models to predict primary energy consumption for space cooling/heating, and got high accuracy of more than 95%.Although these previous study has achieved high accuracy in the task of predicting energy usage, limited research has been done to understand the energy efficiency, which is developed by wellestablished mathematical methods to help estimate and improve the efficient use of energy [21].[13] made effort to verify the accuracy of the energy performance certificates by refining ANN models and defining Neural Energy Performance Index.It turns a small error in only 3.6% of cases was found.[22] designed an ANN model for predicting heating and cooling loads instand of the average energy efficiency of the building directly.How to understand and predict energy efficiency better needs more exploration.
Also previous research usually need to collect site-specific data such as the human behaviour factors (the number of occupants and the heating setpoint temperature, etc.) and the energy consumption-related indicators (CO2 emissions, walls' solar absorptance, Global Energy Performance Index, etc.).These sitespecific factors are either uncontrollable in practice, and its enlightenment on retrofitting for green energy buildings is limited or these factors are not readily available.To extend our research to a larger study area and have a holistic city-level understanding of building energy efficiency, we will not use these factors but only choose abundant and easier to obtain characteristics from EPC and street view images to understand the building itself and its basic energy facilities, based on which to accurately estimate their energy efficiency.This approach makes it possible to assess the energy efficiency at a city level with insufficient data on energy consumption indicators.

Using street-level imagery for building stock estimation
With the increasing coverage of GSV images and computational power, many research have been devoted to combining deep learning and GSV for building stock prediction.GSV is a new source of large-scale urban data that has been widely used in many urban research fields, such as urban planning and design [23,24], real estate [25], urban morphology [26,27], transportation and mobility [28], socio-economic studies [29,30], and urban perception [31].[32] reviewed approximately 600 papers published between 2005 and 2020 using street-level imagery as a research data source, where GSV was used as the data source for two-thirds of the overall papers.The widespread use of GSV is mainly due to its large coverage all over the world (over 200 countries) and standard data quality.
The buildings in GSV are labelled with geographical location, style, age, façade material, volume and scale.With these tags, computer vision algorithms can screen out recognizable image information through various discriminant methods, thus analysing the city and its architectural culture.Based on deep learning for GSV image feature extraction, studies on architecture have developed from being limited to the study of the building itself (e.g., architectural style, age, façade material, volume, and scale) to the study of the area including the buildings.Some studies have investigated certain characteristics of the areas in which they are located by identifying multiple features of buildings and non-architectural factors (e.g.vegetation), such as street space quality [33], urban aesthetics [34,35], continuity of street architecture [36], urban canyon geometry [27], and urban architectural landscape characteristics [37,38].The relationship between building and energy has also received much scholarly attention based on the application of GSV images and deep learning.On the basis of the GSV images of Victoria, Australia, [30] estimated the year of buildings and constructed a dataset of relevant attributes according to GSV images from Victoria, thus providing key information for energy demand and retrofitting of buildings.[39] used GSV and machine learning to predict building features relevant to energy retrofitting (i.e., building type and suitability for additional façade insulation) [39].

Study area
We take Glasgow city in Scotland as the study area.Fig. 1 presents the footprints of domestic buildings with EPC data in our study area.According to Koppen-Geiger classification, the climate of Glasgow is ''Cfb, Marine West Coast Climate" [40].Also, the indoor comfort will be affected by the global warming [41].Besides the climate change issues in Glasgow, as one of the largest cities in the UK, the diversity of building styles, the historical city development, the ambitious goal to achieve a net-zero target by 2045, and the well organized public Scottish EPC dataset, make it an ideal site for building energy efficiency studies.

Data
This study incorporates multi-source data from the Scottish EPC data [42], UKBuildings dataset from EDINA Geomni Digimap Service [43], and GSV images for estimation of building energy efficiency.

EPC data
In the UK, a building must obtain and has an EPC in the past 10 years when it has been newly constructed or is to be sold or rented, except for very few special cases [44].An EPC includes information on the energy efficiency of buildings.It records specific information such as the size and layout of the building, how it has been constructed and the way it is insulated, heated, ventilated, and lighted.Based on these records, the EPC evaluators use a UK government calculation methodology to estimate monthly energy usage and CO2 emissions of buildings and generate the ''energy efficiency rating" of the building from A to G, with A being the best, which can help understand how much fuel cost may need to be paid.
We select 168,410 EPC records for domestic buildings from October 2012 to March 2021 in Glasgow.The data is requested by setting the range of zip code of Scotland and converting addresses (contain street name, street number and zip code) of records into coordinates by Google Geocoding API.After filtering out the records whose coordinates fall outside the study area, we get 165,318 records.This nearly two percent loss may be due to inaccurate or incomplete address information of the EPC dataset.The specific domestic building locations are shown in Fig. 1.After deleting the features that have overlapping descriptions or are obtained by calculation, the numerical features in the EPC we used to predict energy efficiency are shown in Table 1.It includes the building construction factors and facilities.Besides, details about categorical features are presented in the Supplementary material.

UK buildings
UK Buildings dataset provides 2D building footprints across Great Britain for residential, non-residential and mixed-use properties for towns having more than 10,000 population [43].We use this dataset as a geo-referenced dataset to match each EPC record with a street view image that describes its appearance.

Street view images
As the digital representation of the built environments, street view image is a valuable resource for understanding and analyzing architecture and cities.With growing coverage and more providers, the street view image service has covered more than half the world's population [45].It provides a more intuitive and human-perspective view than other data sources.Among all the services, GSV is one of the most popular sources.We obtain images from the GSV service with its own Application Programming Interface (API) with our customized parameters 1 .The detailed calculation of parameters will be presented in methods section 3.3.1.We request GSV for 165,318 properties and download more than 550,000 street view images for 157,222 properties for further analysis.For each property, historical GSV images are also obtained to enlarge the dataset.The dates of images range from 2008 to 2021.Considering that GSV images captured in consecutive years might be too similar and lead to the overfitting of our model, we filter images to make sure the capture year of the historical images of the same property has at least two years gap.As a result, we link 368,769 GSV with the EPC records.

Methodology
To demonstrate how our framework, this section presents the workflow of our methodology: 1) Street view image collection, 2) Model design, 3) Model evaluation, and 4) Model interpretation.The nomenclature and abbreviations are shown in Table 2.The methodology code can be found on the project GitHub repository (https://github.com/MaoranSun/buildingEnergyEfficiency).

Street view image collection
GSV images are available for download via Google API.However, the default view is pointing at the direction along the streets.For this analysis, an image facing the building façade is required.The GSV API allows us to pass customized parameters including heading (the direction the camera is pointing at), field of view (FOV, zoom level of the camera), and pitch (vertical angle of the camera relative to the street view vehicle).Here, we present an algorithm for calculating these parameters below.
Heading represents the direction in which the camera is point- Equation (2) shows the calculation of the pitch angle.
FOV is based on the width of the façade and the distance between the building façade and the requested point.This can be measured with angle b, which is the angle between V SF1 ! and V SF2 ! as illustrated in Fig. 2. As GSV API accepts the value between 0 and 120, we pass the value of 120 or the calculated angle, whichever is smaller.

Multi-branch deep learning model design
With building façades and energy performance descriptive data, we design a deep convolutional neural network for classifying the energy efficiency.This model aims at learning energy efficiency information simuteously from both façade image and property's morphological and structural attributes.As shown in Fig. 3, the network mainly consists of four stages: input, feature extraction, feature fusion and output.In input and feature extraction stages, the network runs in two parallel branches: image branch and descriptive feature branch.The image branch takes building façade photos as input, with Dense Convolutional Network (DenseNet) as backbone and outputs a 1024-dimensional deep feature for feature fusion.Four dense blocks are used in the image branch.For the descriptive feature branch, we build a simple fully-connected neural network with four hidden layers which yields a 256dimensional deep feature.In the feature fusion stage, the 1024-  dimensional feature from the image branch and 256-dimensional feature from the descriptive feature branch are concatenated as a 1280-dimensional deep feature.This concatenated deep feature is then passed into a fully-connected neural network with two hidden layers and makes the final classification of the property energy efficiency categories.
For model initialization, we apply different strategies for the image and descriptive feature branches respectively.The image branch is initialized with weights pre-trained on the ImageNet dataset [46].ImageNet is a widely used dataset for detecting common objects such as vehicle, building and street sign.The pretrained weights used is able to understand and extract information about objects and scenes within the images.This initialization strategy helps the faster convergence of the network and requires less training time.For the descriptive feature branch, we apply random initialization because of the small number of parameters involved and the simple structure.Both characteristics make the branch easier to train.

Evaluation of result
For the main model, the dataset is splitted into three parts: 70% of it is used for training, 15% for validation and 15% is used to test the model.The model is evaluated with several metrics with the test dataset.The evaluation aims at providing details about the performance and where the misclassification happens.Firstly, the model is assessed with numerical metrics of Precision, Recall and F1-score.Besides, we present a confusion matrix for detailed performance of all classes and corresponding numerical metrics.Precision is the ratio of correctly predicted positive samples to all the predicted positive samples as shown in equation (3); Recall is the ratio of correctly predicted positive samples to all the samples in actual classes as shown in equation ( 4); F1-score refers to the harmonic mean of precision and recall as shown in equation (5).Besides the main model, we also apply a 10-fold cross-validation on the full dataset to test the stability of our method with averaging the total accuracy from 10 models.

Precision
Secondly, we map the prediction results to explore the spatial distribution of the model performance, as the adjacent buildings often share similar style, age and façade materials.It is also important to identify the poor performance areas spatially for further improvement.Though the prediction is made at the property level, we aggregate the predictions results to 150-meter grids for two reasons.Firstly, the development and implementation of policies are usually at higher spatial scale rather than property-level.Secondly, it is more intuitive and could better convene the spatial distribution of results.The performance of grid is evaluated with the ratio of correctly classified sample numbers to the total sample number associated within the grid.

Model comparison and interpretation
Besides the multi-branch model, we also predict the building energy efficiency rating from image or descriptive features, respectively.Same as the image branch, the image classification model is adapted from DenseNet architecture with four dense blocks, and yields final prediction for seven rating classes.The descriptive feature is also fed into a simple neural network with four hidden layers.To make sure the results are comparable and avoid the randomness during experiments, we also keep the same split of training, validation and test dataset across models.
To better understand and improve the building energy efficiency, we explore the attributes of property contributing to the model decisions.The interpretability of deep learning has been widely studied recently.We apply the methods proposed by [47] to the multi-branch and descriptive models.For the multi-branch model, we mainly focus on the most decisive regions within the image.For the descriptive models, we explore which descriptive attributes are more important for improving the building energy efficiency.We calculate SHAP2 values for each pixel within images and attributes of properties.SHAP value is the method based on game theory and used to increase transparency and interpretability of our model.More specifically, SHAP values measure the contribution of the factors to the final prediction, with greater value leading to the prediction and smaller value contributing to other possible predictions.

Model performance
With the data and methodology above, we implement the model on the Ubuntu platform with four GeForce RTX 2080 Ti GPUs and Python and PyTorch framework.The model is trained with hyperparameters of 0.005 as the learning rate and 100 as batch size.The training process takes 13.26 h and 45 epochs by using the training (70%) and validation (15%) dataset.As shown in Fig. 4, the validation accuracy becomes stable after the 37th epoch.The final model is evaluated on the held test data and the inference time is 133 samples per second.We evaluate the final model with overall precision, recall and F1-score by categories and spatial distribution of the prediction accuracies.

Overall performance
The final model achieves an overall accuracy of 86.8% on test set.We also test the model performance with 10-fold crossvalidation on the full dataset, results show that our model is able to achieve the mean accuracy of 86.4%.To further explore the detailed performance by classes, we present a confusion matrix containing normalized performance, recall, precision and F1scores for each category.As shown in Table 3, the top-left to bottom-right diagonal shows the percentage of correctly predicted samples over all samples.The off-diagonal space represents the percentage of misclassified samples.Result shows that the model is able to archieve more than 80% accuracy for most classes.For Grade A properties, the model achieves 69% accuracy as shown in Table 3.This is due to the extremely small number of samples from class A compared to that of other categories.It is also worth noticing that most of the confusion happens within adjacent classes.For classes with accuracy under 80%, Grade F samples are often misclassified as Grade E and Grade A samples are mostly misclassified as Grade B and C.

Spatial distribution of prediction
To further understand the framework performance and the characteristics of energy efficiency, we plot the error into the map to explore the spatial distribution of prediction error.Fig. 5 shows the prediction accuracy aggregated into 150-meter grids.The performance is measured by the precision, which is the number of correctly classified samples divided by all associated sample number.The color indicates the accuracy with blue representing high accuracy and magenta showing low accuracy.As shown in the map, the majority of grids achieves a precision of over 80%.Few grids have low precisions and no evident spatial patterns are shown, which indicates no significant spatial autocorrelation exists for the model's outputs (Global Moran's I = 0.14).

Model comparison
As an exploration, we trained two individual models predicting building energy efficiency rating from façade images and descriptive features respectively.The two models are then evaluated based on the same metrics for feature-fusion model.Table 4 presents the accuracy, precision, recall and F1-score for comparisons of different models.As shown in the table, the image feature model and descriptive feature model achieves the accuracy of 57.2% and 79.5%, respectively.It is not surprising that the image feature performs the worst, as the image data is just the external appearance at the building level and the information is insufficient for energy efficiency prediction.Results also shows that the feature-fusion model achieves the highest accuracy.Furthermore, we present the confusion matrix heatmap for each model in Fig. 6 to inspect the break down of performance in categories.The heatmap shows that, the feature-fusion model not only has the best overall accuracy, but also has a more balanced performance across classes.Similar to the feature fusion model, the traditional descriptive feature model has a very low accuracy for Grade A.

Model interpretation
We calculate the SHAP value for multi-branch model and descriptive model respectively with the SHAP Python library developed by [47].Fig. 7 shows the top important features in the descriptive model.We implement KernelExplainer from the SHAP library to explore the impact of each feature to building energy efficiency.Color represents the original value of the feature and X position shows the SHAP value for features.Since we preprocess the data with one-hot encoding, in most cases pink means True while blue represent False.The plot is ordered by the absolute mean of each feature's SHAP values, which could be treated as a proxy for feature importance.Features with high importance are ranked on the top of the figure.SHAP values on the X-axis represent the feature's contribution to energy efficiency, with larger value meaning feature contributes to lower building energy efficiency.Take the feature Roof: Pitched, no insulation for example, pink dots (meaning the property has pitched and non-insulated roof) have high SHAP values, thus contributing to low energy efficiency.The result aligns with the intuition in many ways.'Insulation' plays an important role in energy efficiency.Roof and wall with 'no insulation' have negative impact to the energy efficiency, while insulated wall improves the efficiency.Furthermore, the plot is able to compare the contribution of each feature to identify useful elements for energy efficiency improvement.For example, houses featured with long history are associated with low efficiency.Construction year before 1919 has more negative impact to energy efficiency than construction age band within 1930-1949.
Fig. 8 shows the informative regions with the building façade images.We select random samples from the dataset and calculate the SHAP value for each pixel within images.Pink dots represent the areas with high SHAP value and are important for final decisions.As shown in the plot, most pink dots distribute around structural elements in the building façades, such as windows and doors.This reveals that the model is able to make decisions based on meaningful areas of the building façades, rather than paying attention to random parts.

Building energy efficiency estimation in the era of big data
With the approaches in big data era, more and more data have been generated and made publicly available.This research demonstrates the potential of utilizing publicly available administrative data to estimate building energy efficiency.These data are able to provide extra information for building energy studies and fill the gaps for the traditional energy efficiency estimation methods.This paper recognizes that GSV is informative not only for attributes such as building age and style, which are directly related to the visual aspects of the buildings, but also can extend our understanding of building intrinsic characteristics such as building energy efficiency.As a urban big data source, GSV is able to provide extra information in addition to the traditional building morphological attributes, and achieves a more accurate and holistic description of buildings.With the combination of GSV and EPC,  we are able to achieve high performance for property-level building energy efficiency estimation and prediction.Furthermore, with the increasing number of research on deep learning's interpretability, we are able to find the meaningful features of building morphological attributes and decisive part of building façades for energy efficiency estimation.With the wider enforcement and coverage of EPC data and street view images, this approach s shows greater potential for the future work and can be extended to other cities and countries.

Policy implications
Predicting and understanding the energy efficiency is crucial to the policy making and implementations.As the building sector being the largest energy consumer [48], improving building energy efficiency is effective for greenhouse gas emission reduction, climate change prevention, and carbon-neutral policies.Accurate estimations and comprehensive understanding of property-level energy efficiency ratings are beneficial for regulation and policy making.It has been proven that energy efficiency rating is related to fuel poverty problem [6].Furthermore, it is critical to the implementation of policies.For example, UK government is aiming at increasing as many private rented properties to EPC Band C by 2030 [49].For property owners, it is important to understand not only current ratings, but also how the renovation will affect the energy efficiency for their properties to be legally listed.For policy execution, it also helps to identify the properties which require further renovation and improvement.
The proposed analytical framework in this research is beneficial to both policy making and implementation.The high accuracy of methods ensures the framework could be applied to practice and has the potential to be extended to other cities.The fine-scale property-level prediction makes it possible for home owners to better understand their ratings.Besides, because of the fine scale of the methods, it is easier for city administrations to aggregate the results and set goals in different spatial units (e.g.neighborhood) for better execution, particularly for the deprived neighborhood.Furthermore, our framework takes detailed inputs about the properties such as window description and wall description.It helps the homeowners to estimate the final energy efficiency before the renovation starts.

Limitations and future work
Limitations do exist in this study and could be improved in the future studies.The main limitation of this work is the data quality of EPC dataset.It has been widely disucssed that some uncertainties exist in the EPC dataset in terms of the gap between estimated and actual energy performance [50,51].With the growing coverage of EPC, European countries have built standard for quality assurance.In the future work, the uncertainty of dataset can be gradually minimized.
Secondly, the detailed attributes of EPC dataset also constrain the widely application of our framework.Most of the descriptions about properties from EPC could not be obtained through other data sources.It is very difficult to make good prediction for buildings with general building information (i.e.building height, age) outside of EPC database.For future work, we plan to explore the balance between the data availability and prediction accuracy, so that the framework could be extended more broadly to demostic buildings without EPC now.At present, the coverage of EPC data is about 50% in England, Wales, and Scotland [52], and a large number of properties that have GSV does not have EPC data.We can use data from other open dataset instead of EPC as the traditional fea- tures in the framework to predict fine-grained city-level or largerscale energy efficiency and gain insight into energy efficiency differences among different regions.

Conclusion
Improving building energy efficiency is key to the global carbon emission reduction task.Accurate predicting and understanding of building energy efficiency is beneficial for better utilizing and saving energy in the building sector.This paper proposes a featurefusion framework for building energy efficiency prediction with publicly available data.The framework involves EPC data collection of building descriptive factors and street-level imagery data, and we extract and fuse the features from both data sources for the final estimation.The framework is implemented for the city of Glasgow, UK for its feasibility.Results show that our framework is able to correctly classify 86.8% samples from test set.With the comparison of our feature-fusion framework, image-only model and traditional descriptive factors model, our framework is able to achieve the highest accuracy and has a more balanced performance across different ratings.The explainable AI tool indicates that insulations around open structures such as windows and doors are key factors to influence the energy efficiency.
Our research contributes to the research of building energy studies in twofold.First, by incorporating street view images, for energy efficiency estimation task we are able to achieve higher accuracy compared to traditional building attribute features.Second, our method is able to identify important building features to improve building energy efficiency, which will be useful for the housing retrofit in the near future.This study also provides insights into the potential of applying deep learning to the research of building attributes with new forms of urban big data.By introducing street view images to the building energy studies as a visual representation and proxy for building ages, styles and façade materials, we verify that GSV is able to provide another layer for build- ing stock attributes prediction.This research has the potential to help urban planners and policy makers to target specific 'energy efficiency deprived' neighborhood and provides extra evidence to better tackle the fuel poverty problems efficiently.

ð1Þ
ing at.More specifically, what the parameter needs is the angle a between the Vector North and Vector SC, as shown in Fig. 2. Equation (1) shows the calculation of the heading parameter from Point S x s ; y s ð Þ, Point C x c ; y c ð Þand Vector North.It is worth noticing that angle a is the clockwise rotation angle from Vecttor North to Vec-Pitch controls the vertical direction of the camera.This could be calculated based on the angle c between Vector SC and Vector ST.

Fig. 3 .
Fig. 3. Deep learning model architecture.The model takes image and descriptive features as input, processes two branches simultaneously and make a final prediction.

Fig. 4 .
Fig. 4. Model training curve (green dash line: validation accuracy; blue line: model accuracy).(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 7 .
Fig. 7. Feature Importance in descriptive attributes.The features from top to bottom have decreasing feature importance.

Fig. 8 .
Fig. 8. Feature Importance in image branch of multi-branch model.Pink dots represent areas important for final decision.(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Table 1
Numerical EPC Features used in the analysis.

Table 2
Nomenclature and Abbreviations.

Table 3
Confusion matrix for energy efficiency rating classification.

Table 4
Model performance comparison.