Simultaneous Capturing of LiDAR and Imagery

Author: Mathias Lemmens

People continue to migrate from rural areas to major cities, driving sustained urban growth and increasing the demand for accurate, detailed and up-to-date 3D city models. The creation of such models is still a cumbersome endeavour, but new advancements, such as the combination of three sensor types — nadir camera, oblique cameras and a LiDAR unit — in one and the same geodata acquisition system, may bring relief. Aerial surveys conducted in major cities in the UK and Ireland demonstrate the potential of this solution.

Cities will continue to grow as long as the world’s population keeps flocking to urban areas. In western countries, migration from rural to urban areas started in the early 1800s, and by the early 1900s 15% of the global population was living in cities. This will have increased to 60% by 2030, thanks in part to the large-scale shift toward urban living in Africa and Asia, which began in the mid-1990s. This population concentration has created dozens of urban agglomerations with more than 10 million inhabitants, called megacities. According to a study of the world’s demography by Euromonitor International, in the decade between 2020 and 2030 six new megacities will emerge: Chicago, Bogota, Luanda, Chennai, Baghdad and Dar es Salaam. By 2030 there will be 39 megacities, which between them will be home to nearly 10% of the population and produce 15% of the world’s gross domestic product (GDP). The liveability and sustainability of megacities rely on well-functioning roads, subways, railways, bridges, schools, hospitals and other public services.

3D City Models

Most modern cities and megacities have become complex, multifaceted 3D landscapes. Those responsible for the management, security and development of these major cities require detailed 3D models of buildings and infrastructure to support them in their tasks. 3D models are usually produced manually from stereo photogrammetry. Accurate and detailed 3D city models are in high demand, but expensive to produce. Realistic views require rendering with images, which is a labour-intensive activity. Often 3D city models consist of rendered polygon meshes that are common in computer graphics, gaming and animation. For 3D city modelling, mesh models are used for visualisation, line-of-sight analysis, risk assessment, noise modelling, flood modelling, master planning and much more. They are also indispensable for creating smart cities.

Mesh Models

A mesh is a group of edges, lines and faces that define the surface shape of a 3D object. The faces often consist of a network of triangles, usually referred to as a triangular irregular network (TIN). The higher the point density is, the denser the mesh and the more detailed the representation of the 3D object will be. If the points constituting the triangles are represented in the same reference system as images, photorealistic 3D objects can be combined to create an entire city highly automatically. Airborne LiDAR and vertical and oblique aerial images are well suited to the creation of 3D city models. Mesh models are often regarded as purely visual 3D models. However, limiting their use to visualisation purposes alone would do an injustice to their full potential. Particularly when georeferenced at decimetre accuracy or even higher, such 3D data can be used for measuring distances, heights, surfaces and volumes. In addition, it allows line of sight and other types of analysis, shading and flood modelling. Once it is known that a collection of adjacent meshes form a building or other coherent object, it is possible to assign an address, market value, BIM information or other information to the conglomerate of meshes. Attributing semantic information allows queries and intelligent analysis to be conducted.

Bottlenecks

There are several bottlenecks in the creation of 3D mesh models of cities. One of the bottlenecks concerns the acquisition of homogeneous data over the entire survey area. Point clouds generated from images show impediments, even if the images are acquired with high overlaps and using dense image matching techniques, resulting in the extraction of no — or unreliable — points for some areas. Reasons for these shortcomings include:

Occlusion: in narrow streets or urban canyons, some objects may block the view of other objects such as building facades. To extract 3D points from images, it is a fundamental requirement that objects are visible in at least two images.

Presence of shadows: although dense image matching based on semi-global matching is highly robust against the absence of edges and texture, it may still cause unreliable matches.

Today’s airborne LiDAR systems are able to create point clouds with high point density. However, the points have only one spectral value, which is the intensity of the return pulse. Nevertheless, the returns are not affected by the presence of shadows caused by sunlight, which is a clear advantage compared to photogrammetry. Since objects only have to be visible from one viewpoint, airborne LiDAR is less affected by occlusion than photogrammetry.

Leica CityMapper

To tackle the above-mentioned bottlenecks of photogrammetry on the one hand and airborne LiDAR on the other, Leica Geosystems has combined three sensor types in the world’s first hybrid airborne geodata acquisition system. Called CityMapper, this system combines – in a single pod – one RCD30 CH82 multispectral camera for capturing nadir imagery, four RCD30 CH81 m cameras for capturing oblique imagery and one LiDAR unit. The nadir and oblique-looking heads are arranged according to the Maltese cross concept. The images of nadir camera captures RGB and the near-infrared (NIR) at 0.78 to 0.88 µm. T

he four oblique cameras capture RGB imagery at 45 degrees forwards, backwards, left and right. The nadir imagery can be captured at a ground sample distance (GSD) of 3 centimetres with a potential accuracy of 6 cm root mean square error (RMSE). At the centre of the image, the GSD of oblique images is approximately 75% of the nadir GSD due to a combination of looking angle and focal length. When the nadir imagery has a GSD of 5 cm, the GSD of the oblique imagery at the centre will be 3.7 cm. The LiDAR unit emits laser pulses with a wavelength of 1,064 µm and a pulse repetition frequency of up to 700 kHz. The accuracy is 6 cm and the point density is 15 points per square metre at a flying height of 750 metres. None of these three sensor types nor their specifications are new.

The workflows of generating outputs from the three data types are well established and operational in many photogrammetric companies. What is new is that incorporation in a single pod and using a single control unit enables the imagery and LiDAR point clouds to be acquired simultaneously. This simultaneous data acquisition offers many advantages for the creation of 3D city models. For example, LiDAR pulses are able to penetrate street canyons where there may be shadows due to obscured sunlight, and in narrow streets – where occlusion might prevent cameras from obtaining two views — LiDAR can double the chance of successful data capture.

Use cases

CityMapper has been extensively deployed in Asia for accurate and detailed 3D mapping of new megacities with huge skyscrapers, which have been constructed at a breath-taking rate since 2000, and interest is now growing in Europe, too. In 2018 and 2019, U.K.-based aerial mapping company Bluesky captured parts of London, Manchester, Birmingham, Cambridge, Oxford and several other U.K. cities. The nadir and oblique images as well as the LiDAR point clouds are processed using HxMap, which contains tools for data download and raw quality control. Additional tools for aerial triangulation, radiometric adjustments and point cloud registration and georeferencing are available. Subsequently, the data is further processed to digital surface models, digital terrain models, orthoimages and other products derived from CityMapper imagery and LiDAR point clouds, which Bluesky calls ‘MetroVista.’ As part of a major transport infrastructure project, a photorealistic MetroVista 3D mesh model was used to gain insight into the impact of the proposed construction by adding the object to the accurate 3D mesh model at the exact location and orientation. The model of the existing real world combined with the proposed development allowed the nature and scale of the development to be communicated to stakeholders and the general public.

Challenges

Needless to say, the creation of sophisticated, high-accuracy products that benefit city governments, managers and planners alike is not without its challenges. An aerial photogrammetric survey produces large data volumes that require parallel processing and hence significant investment in hardware and software in order to process, disseminate and share them efficiently and reliably. Other challenges include the weather in the U.K., which is often not ideal for conducting aerial surveys. The flying height — between 1,000 m and 1,800 m — is lower than standard aerial surveys, which can be beneficial since it is often below cloud cover. However, the lower flying height can also draw increased attention from air traffic control in the crowded airspace above cities and megacities, leading to delays. Moreover, the flying speed — which at around 220 km/h is slightly slower than for traditional survey flights — increases the data capture time, which may cause issues with permissions or weather windows.

This story first appeared in GIM International — https://www.gim-international.com/content/article/simultaneous-capturing-of-lidar-and-imagery

Major efficiency improvement for airborne urban mapping solution

In September 2019, Leica Geosystems announced a significant upgrade to the Leica CityMapper sensor system, called the Leica CityMapper-2. This next-generation hybrid oblique imaging and LiDAR sensor was first delivered to customers in June 2020. Providing fast and efficient digitisation of cities, the latest version of the sensor was designed to provide faster updates while preserving image quality over a wide range of flying conditions. The CityMapper-2 comes with a newly developed highperformance optical system incorporating two nadir (RGB & NIR) and four oblique 150 MP metric cameras using CMOS technology and equipped with Leica Geosystems’ unique mechanical forward-motion-compensation (FMC). The LiDAR sensor’s pulse repetition frequency has been increased to 2 MHz and features gateless Multiple-Pulsesin-the-Air (MPiA) technology.

For more information, visit leica-geosystems.com/citymapper-2