ICEYE is a 10-year-old manufacturer and operator of a Synthetic Aperture Radar (SAR) satellite fleet. They have a headquarters in Espoo, Finland, about a 25-minute drive from Helsinki city centre as well as offices in Poland and the UK.
ICEYE's satellites can see through clouds, smoke, rain, snow and certain types of camouflage attempting to cover the ground below. They can capture images day or night at resolutions as fine as 50cm.
False-colour techniques can be applied to SAR imagery to help highlight man-made objects. Below is an image of Tesla's Gigafactory in Austin, Texas that ICEYE captured recently.
SAR collects images via radar waves rather than optically. The resolution of the resulting imagery can be improved the longer a point is captured as the satellite flies over the Earth. This also means video is possible.
Below is the first few seconds of a 12-second, 4K video of Hamad International Airport in Doha, Qatar. It was taken by their ICEYE-X20 satellite on April 30th of this year.
ICEYE has launched 39 satellites into space with only an astounding single failure. This fleet can revisit locations as frequently as every six hours.
Below is an artist's depiction of one of ICEYE's satellites.
Their first five launches were split across four different launch partners and countries. But from their sixth launch onward, they've been launching exclusively with SpaceX from the US.
I haven't been able to identify the satellite numbers for their last seven deployments. If I come across these details, I'll update this post.
Launch # | Launch Date | Satellite | Launch Vehicle | Launch Site
------------------------------------------------------------------------------------------------------------------------------
1 | 2018-01-12 | ICEYE-X1 | PSLV-XL | Satish Dhawan Space Centre, India
2 | 2018-12-03 | ICEYE-X2 | Falcon 9 Block 5 | Vandenberg Space Launch Complex 4 (SLC-4)
3 | 2019-05-05 | ICEYE-X3 | Rocket Lab Electron | Launch Complex 1, New Zealand
4 | 2019-07-05 | ICEYE-X4 | Soyuz-2-1b | Vostochny Cosmodrome Site 1S, Russia
4 | 2019-07-05 | ICEYE-X5 | Soyuz-2-1b | Vostochny Cosmodrome Site 1S, Russia
5 | 2020-09-28 | ICEYE-X6 | Soyuz-2-1v | Plesetsk Cosmodrome, Russia
5 | 2020-09-28 | ICEYE-X7 | Soyuz-2-1v | Plesetsk Cosmodrome, Russia
6 | 2021-01-24 | ICEYE-X8 | Falcon 9 Block 5 Transporter 1 | Vandenberg Space Launch Complex 4 (SLC-4)
6 | 2021-01-24 | ICEYE-X9 | Falcon 9 Block 5 Transporter 1 | Vandenberg Space Launch Complex 4 (SLC-4)
6 | 2021-01-24 | ICEYE-X10 | Falcon 9 Block 5 Transporter 1 | Vandenberg Space Launch Complex 4 (SLC-4)
7 | 2021-06-30 | ICEYE-X11 | Falcon 9 Block 5 Transporter 2 | Cape Canaveral Space Force Station in Florida (SLC-40)
7 | 2021-06-30 | ICEYE-X12 | Falcon 9 Block 5 Transporter 2 | Cape Canaveral Space Force Station in Florida (SLC-40)
7 | 2021-06-30 | ICEYE-X13 | Falcon 9 Block 5 Transporter 2 | Cape Canaveral Space Force Station in Florida (SLC-40)
7 | 2021-06-30 | ICEYE-X15 | Falcon 9 Block 5 Transporter 2 | Cape Canaveral Space Force Station in Florida (SLC-40)
8 | 2022-01-13 | ICEYE-X14 | Falcon 9 Block 5 Transporter 3 | Cape Canaveral Space Force Station in Florida (SLC-40)
8 | 2022-01-13 | ICEYE-X16 | Falcon 9 Block 5 Transporter 3 | Cape Canaveral Space Force Station in Florida (SLC-40)
9 | 2022-05-25 | ICEYE-X17 | Falcon 9 Block 5 Transporter 5 | Cape Canaveral Space Force Station in Florida (SLC-40)
9 | 2022-05-25 | ICEYE-X18 | Falcon 9 Block 5 Transporter 5 | Cape Canaveral Space Force Station in Florida (SLC-40)
9 | 2022-05-25 | ICEYE-X19 | Falcon 9 Block 5 Transporter 5 | Cape Canaveral Space Force Station in Florida (SLC-40)
9 | 2022-05-25 | ICEYE-X20 | Falcon 9 Block 5 Transporter 5 | Cape Canaveral Space Force Station in Florida (SLC-40)
9 | 2022-05-25 | ICEYE-X24 | Falcon 9 Block 5 Transporter 5 | Cape Canaveral Space Force Station in Florida (SLC-40)
10 | 2023-01-03 | ICEYE-X21 | Falcon 9 Block 5 Transporter 6 | Cape Canaveral Space Force Station in Florida (SLC-40)
10 | 2023-01-03 | ICEYE-X22 | Falcon 9 Block 5 Transporter 6 | Cape Canaveral Space Force Station in Florida (SLC-40)
10 | 2023-01-03 | ICEYE-X27 | Falcon 9 Block 5 Transporter 6 | Cape Canaveral Space Force Station in Florida (SLC-40)
11 | 2023-06-12 | ICEYE-X23 | Falcon 9 Transporter 8 | Vandenberg Space Launch Complex 4 (SLC-4E)
11 | 2023-06-12 | ICEYE-X25 | Falcon 9 Transporter 8 | Vandenberg Space Launch Complex 4 (SLC-4E)
11 | 2023-06-12 | ICEYE-X26 | Falcon 9 Transporter 8 | Vandenberg Space Launch Complex 4 (SLC-4E)
11 | 2023-06-12 | ICEYE-X30 | Falcon 9 Transporter 8 | Vandenberg Space Launch Complex 4 (SLC-4E)
12 | 2023-11-11 | ICEYE-X31 | Falcon 9 Transporter 9 | Vandenberg Space Launch Complex 4 (SLC-4E)
12 | 2023-11-11 | ICEYE-X32 | Falcon 9 Transporter 9 | Vandenberg Space Launch Complex 4 (SLC-4E)
12 | 2023-11-11 | ICEYE-X34 | Falcon 9 Transporter 9 | Vandenberg Space Launch Complex 4 (SLC-4E)
12 | 2023-11-11 | ICEYE-X35 | Falcon 9 Transporter 9 | Vandenberg Space Launch Complex 4 (SLC-4E)
13 | 2024-03-04 | ICEYE-X?? | Falcon 9 Transporter 10 | Vandenberg Space Launch Complex 4 (SLC-4E)
13 | 2024-03-04 | ICEYE-X?? | Falcon 9 Transporter 10 | Vandenberg Space Launch Complex 4 (SLC-4E)
13 | 2024-03-04 | ICEYE-X?? | Falcon 9 Transporter 10 | Vandenberg Space Launch Complex 4 (SLC-4E)
14 | 2024-08-16 | ICEYE-X?? | Falcon 9 Transporter 11 | Vandenberg Space Launch Complex 4 (SLC-4E)
14 | 2024-08-16 | ICEYE-X?? | Falcon 9 Transporter 11 | Vandenberg Space Launch Complex 4 (SLC-4E)
14 | 2024-08-16 | ICEYE-X?? | Falcon 9 Transporter 11 | Vandenberg Space Launch Complex 4 (SLC-4E)
14 | 2024-08-16 | ICEYE-X?? | Falcon 9 Transporter 11 | Vandenberg Space Launch Complex 4 (SLC-4E)
In this post, I'll examine ICEYE's 18K-image archive and try detecting aircraft in some of their imagery.
Aircraft in SAR Imagery
Aircraft at known airports transmitting ADS-B messages do a good job of telling the world where they are. Paying thousands for this sort of SAR imagery doesn't make a lot of sense. However, aircraft that have turned their ADS-B transponders off and are parked on a public highway are a different matter.
To add to this, it's rare in Northern Europe to have a clear sky regardless of the time of year. Clouds can be a real challenge here.
That said, some aircraft can be difficult to spot in SAR imagery. Below is Esri's satellite image of an Aircraft Boneyard.
Below is Umbra's (one of ICEYE's competitors) image of the same location. Though it was taken on a different day and some aircraft might have been moved around, you can see that a lot of the aircraft in the bottom left are barely visible unless you zoom in very closely and pay attention to artefacts that give away a large man-made object is present.
SARDet-100K & MSFA
In March, a paper was published titled "SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection". It describes a new dataset called SARDet-100K and the Multi-Stage with Filter Augmentation (MSFA) framework for detecting objects in SAR imagery.
The dataset is made up of 117K SAR images with 246K objects annotated within them. These were sourced from ten existing datasets. The annotations highlight aircraft, bridges, cars, harbours, ships and tanks.
The framework tries to bridge the gap between RGB and SAR imagery. There are also several tools to help reduce multiplicative speckle noise and artefacts in SAR imagery.
My Workstation
I'm using a 6 GHz Intel Core i9-14900K CPU. It has 8 performance cores and 16 efficiency cores with a total of 32 threads and 32 MB of L2 cache. It has a liquid cooler attached and is housed in a spacious, full-sized, Cooler Master HAF 700 computer case. I've come across videos on YouTube where people have managed to overclock the i9-14900KF to 9.1 GHz.
The system has 96 GB of DDR5 RAM clocked at 6,000 MT/s and a 5th-generation, Crucial T700 4 TB NVMe M.2 SSD which can read at speeds up to 12,400 MB/s. There is a heatsink on the SSD to help keep its temperature down. This is my system's C drive.
The system is powered by a 1,200-watt, fully modular, Corsair Power Supply and is sat on an ASRock Z790 Pro RS Motherboard.
I'm running Ubuntu 22 LTS via Microsoft's Ubuntu for Windows on Windows 11 Pro. In case you're wondering why I don't run a Linux-based desktop as my primary work environment, I'm still using an Nvidia GTX 1080 GPU which has better driver support on Windows and I use ArcGIS Pro from time to time which only supports Windows natively.
Installing Prerequisites
I'll be using Python and a few other tools to help analyse the data in this post.
$ sudo apt update
$ sudo apt install \
gdal-bin \
jq \
libimage-exiftool-perl \
python3-pip \
python3-virtualenv \
tree
I'll be using JSON Convert (jc) to convert the output of various CLI tools into JSON.
$ wget https://github.com/kellyjonbrazil/jc/releases/download/v1.25.2/jc_1.25.2-1_amd64.deb
$ sudo dpkg -i jc_1.25.2-1_amd64.deb
I'll set up a Python Virtual Environment and install some dependencies.
$ virtualenv ~/.msfa
$ source ~/.msfa/bin/activate
$ pip install \
geopandas \
osgeo \
pandas \
rich \
shapely \
typer \
xmljson
I'll be using osm_split to extract geometry and metadata into named GPKG files. These will be used to annotate imagery in this post.
$ git clone https://github.com/marklit/osm_split ~/osm_split
$ pip install -r ~/osm_split/requirements.txt
The following will install some PyTorch dependencies.
$ pip install \
torch==2.0.1 \
torchvision==0.15.2 \
torchaudio==2.0.2 \
--index-url https://download.pytorch.org/whl/cu118
MSFA relies on a number of OpenMMLab packages.
$ pip install -U openmim
$ mim install -U \
mmengine \
mmcv \
mmdet \
mmpretrain
The following will install the MSFA Framework.
$ git clone https://github.com/zcablii/SARDet_100K
$ cd SARDet_100K/MSFA
$ pip install -r requirements.txt
$ pip install -v -e .
I had some issues with SciPy 1.14.1 so I've downgraded to version 1.12.0.
$ pip install -U 'scipy<1.13'
I'll use DuckDB, along with its H3, JSON, Parquet and Spatial extensions, in this post.
$ cd ~
$ wget -c https://github.com/duckdb/duckdb/releases/download/v1.0.0/duckdb_cli-linux-amd64.zip
$ unzip -j duckdb_cli-linux-amd64.zip
$ chmod +x duckdb
$ ~/duckdb
INSTALL h3 FROM community;
INSTALL json;
INSTALL parquet;
INSTALL spatial;
I'll set up DuckDB to load every installed extension each time it launches.
$ vi ~/.duckdbrc
.timer on
.width 180
LOAD h3;
LOAD json;
LOAD parquet;
LOAD spatial;
The maps in this post were rendered with QGIS version 3.38.0. QGIS is a desktop application that runs on Windows, macOS and Linux. The application has grown in popularity in recent years and has ~15M application launches from users around the world each month.
I used QGIS' Tile+ plugin to add geospatial context with Esri's World Imagery and CARTO's Basemaps to the maps.
Downloading Satellite Imagery
ICEYE have a free downloads section on their site but you have to submit your contact and company information via a web form to access their imagery.
I downloaded their "Doha International Airport, Qatar" dataset deliverables as well as the 18K-image thumbnail archive from them.
$ ls -lh Qatar*/*.{tif,mp4}
1.5G .. Qatar_Dwell_Fine_ICEYE_CSI/ICEYE_X20_CSI_SLEDF_4049621_20240429T222841.tif
2.4G .. Qatar_Dwell_Fine_ICEYE_GRD/ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.tif
17M .. Qatar_Dwell_Fine_ICEYE_QUICKLOOK/ICEYE_X20_QUICKLOOK_SLEDF_4049621_20240429T222841.tif
51M .. Qatar_Dwell_Fine_ICEYE_VID/ICEYE_X20_VID_SLEDF_4049621_20240429T222841.mp4
594M .. Qatar_Dwell_Fine_ICEYE_VID/ICEYE_X20_VID_SLEDF_4049621_20240429T222841.tif
$ ls -lh ICEYE_Public_Archive_with_Preview_Images_2020/*.geojson
49M .. ICEYE_Public_Archive_with_Preview_Images_2020/ICEYE_Public_Archive_with_Preview_Images_2020.geojson
ICEYE's GeoTIFFs
ICEYE keep their capture metadata within their GeoTIFFs. Below is the metadata for the Doha imagery.
$ cd ~/Qatar_Dwell_Fine_ICEYE_GRD
$ exiftool ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.tif \
| jc --kv \
| jq -r '."GDAL Metadata"' \
| sed "s/\.$//" \
| xml2json --d yahoo \
| jq '.GDALMetadata.Item[]|{(.name): .content}' \
| jq -s add \
| jq -S --argjson n 50 \
'.[] |= (tostring | if length > $n then .[:$n] + "..." else . end)'
{
"ACQUISITION_END_UTC": "2024-04-29T22:29:03.618788",
"ACQUISITION_ID": "4049621",
"ACQUISITION_MODE": "spotlight",
"ACQUISITION_PRF": "6193.5567",
"ACQUISITION_START_UTC": "2024-04-29T22:28:41.593317",
"AVG_SCENE_HEIGHT": "-22.8837",
"AZIMUTH_DIR": "[-0.117655 -0.442260 0.889136]",
"AZIMUTH_LOOKS": "10.000000",
"AZIMUTH_LOOK_BANDWIDTH": "3158.822346",
"AZIMUTH_LOOK_OVERLAP": "172.211133",
"AZIMUTH_RESOLUTION": "0.454545",
"AZIMUTH_SPACING": "0.250000",
"CALIBRATION_FACTOR": "9.199071e+05",
"CARRIER_FREQUENCY": "9600000000.0000",
"CHIRP_BANDWIDTH": "600000000.0000",
"CHIRP_DURATION": "3.2352e-05",
"COA_POS": "[4082605.980 4680939.303 2850319.325]",
"COA_VEL": "[-900.567 -3404.386 6869.523]",
"COORD_CENTER": "[10000 10887 25.262874 51.571839]",
"COORD_FIRST_FAR": "[1 21774 25.244806 51.600767]",
"COORD_FIRST_NEAR": "[1 1 25.236561 51.551978]",
"COORD_LAST_FAR": "[20000 21774 25.289181 51.591707]",
"COORD_LAST_NEAR": "[20000 1 25.280934 51.542901]",
"DATA_ORIENTATION": "shadows_down",
"DC_ESTIMATE_COEFFS": "[[0.0 0.0 0.0 0.0][0.0 0.0 0.0 0.0][0.0 0.0 0.0 0....",
"DC_ESTIMATE_POLY_ORDER": "3",
"DC_ESTIMATE_TIME_UTC": "['2024-04-29T22:28:52.147512' '2024-04-29T22:28:52...",
"DOPPLER_RATE_COEFFS": "[-61811297623.206978 446504200.214178 -1210045.389...",
"DOPPLER_RATE_POLY_ORDER": "3",
"FIRST_PIXEL_TIME": "0.0035947494",
"FPN": "[0.562089 0.708464 0.426772]",
"GEO_REF_SYSTEM": "WGS84",
"INCIDENCE_CENTER": "32.9348",
"INCIDENCE_FAR": "33.1594",
"INCIDENCE_NEAR": "32.7093",
"LOOK_SIDE": "RIGHT",
"NUMBER_OF_AZIMUTH_SAMPLES": "20000",
"NUMBER_OF_RANGE_SAMPLES": "21774",
"NUMBER_OF_STATE_VECTORS": "32",
"ORBIT_DIRECTION": "ASCENDING",
"ORBIT_PROCESSING_LEVEL": "precise",
"POLARIZATION": "VV",
"POSX": "[4092320.846349 4091734.075728 4091144.583588 4090...",
"POSY": "[4718623.868188 4716289.547421 4713952.423429 4711...",
"POSZ": "[2773337.094165 2778161.314330 2782983.837436 2787...",
"PROCESSING_PRF": "28707.100337",
"PROCESSING_TIME": "2024-04-30T02:11:21.686465",
"PROCESSOR_VERSION": "ICEYE_I_0.9.10",
"PRODUCT_FILE": "ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.tif",
"PRODUCT_NAME": "ICEYE_X20_GRD_SLEDF_4049621_20240429T222841",
"PRODUCT_TYPE": "spotlight",
"RANGE_DIR": "[-0.818665 0.549986 0.165235]",
"RANGE_LOOKS": "1.000000",
"RANGE_LOOK_BANDWIDTH": "600000000.000000",
"RANGE_LOOK_OVERLAP": "0.000000",
"RANGE_RESOLUTION_CENTER": "0.505458",
"RANGE_RESOLUTION_FAR": "0.502422",
"RANGE_RESOLUTION_NEAR": "0.508553",
"RANGE_SPACING": "0.229632",
"SATELLITE_LOOK_ANGLE": "30.489656",
"SATELLITE_NAME": "ICEYE-X20",
"SCENE_CENTER": "[3587255.610427 4521423.223636 2705427.887034]",
"SENSOR_REF_POS": "[4082605.980381 4680939.303280 2850319.324922]",
"SENSOR_REF_VEL": "[-0.116660 -0.441008 0.889889]",
"SLANT_RANGE_TO_FIRST_PIXEL": "540201.0250",
"SPEC_VERSION": "2.5",
"STATE_VECTOR_TIME_UTC": "['2024-04-29T22:28:41.433312' '2024-04-29T22:28:42...",
"TROPO_RANGE_DELAY": "2.904592",
"VELX": "[-838.231191 -842.128106 -846.024914 -849.921612 -...",
"VELY": "[-3340.437760 -3344.452833 -3348.465474 -3352.4756...",
"VELZ": "[6908.873064 6906.445199 6904.013125 6901.576843 6...",
"WINDOW_FUNCTION_AZIMUTH": "NONE",
"WINDOW_FUNCTION_RANGE": "NONE",
"ZERODOPPLER_END_UTC": "2024-04-29T22:28:52.842750",
"ZERODOPPLER_START_UTC": "2024-04-29T22:28:52.146059"
}
The image is delivered with 32 bits per channel.
$ exiftool ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.tif \
| jc --kv \
| jq -S 'del(."GDAL Metadata")'
{
"Bits Per Sample": "32",
"Compression": "LZW",
"Directory": ".",
"ExifTool Version Number": "12.40",
"File Access Date/Time": "2024:09:27 20:38:23+03:00",
"File Inode Change Date/Time": "2024:09:27 11:58:46+03:00",
"File Modification Date/Time": "2024:08:13 15:10:09+03:00",
"File Name": "ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.tif",
"File Permissions": "-rwxrwxrwx",
"File Size": "2.4 GiB",
"File Type": "BTF",
"File Type Extension": "btf",
"Geo Tiff Ascii Params": "(Binary data 7 bytes, use -b option to extract)",
"Geo Tiff Directory": "(Binary data 104 bytes, use -b option to extract)",
"Geo Tiff Double Params": "(Binary data 23 bytes, use -b option to extract)",
"Image Height": "21774",
"Image Size": "20000x21774",
"Image Width": "20000",
"MIME Type": "image/x-tiff-big",
"Megapixels": "435.5",
"Model Transform": "-4.53874229989282e-07 2.24082140708342e-06 0 51.551977778945 2.21873519273927e-06 3.78676304055122e-07 0 25.2365610210872 0 0 0 0 0 0 0 1",
"Photometric Interpretation": "BlackIsZero",
"Planar Configuration": "Chunky",
"Predictor": "None",
"Sample Format": "Float",
"Samples Per Pixel": "1",
"Subfile Type": "Reduced-resolution image",
"Tile Byte Counts": "(Binary data 61 bytes, use -b option to extract)",
"Tile Length": "256",
"Tile Offsets": "(Binary data 67 bytes, use -b option to extract)",
"Tile Width": "256"
}
When I was trying to run inference on these images, OpenCV complained that it didn't support 32-bit samples.
global grfmt_tiff.cpp:710 readData OpenCV TIFF: TIFFRGBAImageOK: Sorry, can not handle images with 32-bit samples
OpenCV can work with either 8 or 16 bits per channel. Below is a GDAL command to convert a 32-bit ICEYE GeoTIFF into a 16-bit one.
$ gdal_translate \
ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.tif \
ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.16bits.tif \
-ot UInt16
GeoFabrik's OSM Partitions
GeoFabrik publishes a GeoJSON file that shows all of the spatial partitions they use for their OpenStreetMap (OSM) releases. I'll use these partitions to help cluster ICEYE's 18K thumbnails by region.
$ wget -O ~/geofabrik.geojson \
https://download.geofabrik.de/index-v1.json
Below is an example record. I've excluded the geometry field for readability.
$ jq -S '.features[0]' ~/geofabrik.geojson \
| jq 'del(.geometry)'
{
"properties": {
"id": "afghanistan",
"iso3166-1:alpha2": [
"AF"
],
"name": "Afghanistan",
"parent": "asia",
"urls": {
"bz2": "https://download.geofabrik.de/asia/afghanistan-latest.osm.bz2",
"history": "https://osm-internal.download.geofabrik.de/asia/afghanistan-internal.osh.pbf",
"pbf": "https://download.geofabrik.de/asia/afghanistan-latest.osm.pbf",
"pbf-internal": "https://osm-internal.download.geofabrik.de/asia/afghanistan-latest-internal.osm.pbf",
"shp": "https://download.geofabrik.de/asia/afghanistan-latest-free.shp.zip",
"taginfo": "https://taginfo.geofabrik.de/asia:afghanistan",
"updates": "https://download.geofabrik.de/asia/afghanistan-updates"
}
},
"type": "Feature"
}
These partitions cover anywhere with a landmass on Earth.
I'll import GeoFabrik's spatial partitions into DuckDB.
$ ~/duckdb ~/iceye.duckdb
CREATE OR REPLACE TABLE geofabrik AS
SELECT * EXCLUDE(urls),
REPLACE(urls::JSON->'$.pbf', '"', '') AS url
FROM ST_READ('/home/mark/geofabrik.geojson');
Below I'll list out all of the partitions and their corresponding PBF URLs containing OSM data for Monaco.
SELECT id,
url
FROM geofabrik
WHERE ST_CONTAINS(geom, ST_POINT(7.4210967, 43.7340769))
ORDER BY ST_AREA(geom);
┌────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────┐
│ id │ url │
│ varchar │ varchar │
├────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ monaco │ https://download.geofabrik.de/europe/monaco-latest.osm.pbf │
│ provence-alpes-cote-d-azur │ https://download.geofabrik.de/europe/france/provence-alpes-cote-d-azur-latest.osm.pbf │
│ alps │ https://download.geofabrik.de/europe/alps-latest.osm.pbf │
│ france │ https://download.geofabrik.de/europe/france-latest.osm.pbf │
│ europe │ https://download.geofabrik.de/europe-latest.osm.pbf │
└────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────┘
Some areas have more than one file covering their geometry. This allows for downloading smaller files if you're only interested in a certain location or a larger file if you want to explore a larger area. Western Europe is heavily partitioned as shown below.
I'll use these partitions to better identify the regions of the world that ICEYE's images are in.
ICEYE's 18K Thumbnails
The thumbnails in the public archive are PNGs devoid of any embedded geospatial or capture data. They are constrained to within 1024x1024 pixels and they've been tinted light green-blue.
The thumbnails are available via HTTPS and their URLs, along with their metadata is kept within a GeoJSON manifest. Below is the first record from this manifest.
$ cd ~/ICEYE_Public_Archive_with_Preview_Images_2020
$ jq -S .features[0] ICEYE_Public_Archive_with_Preview_Images_2020.geojson
{
"geometry": {
"coordinates": [
[
[
116.68105920150816,
-20.67952294953898,
0
],
[
116.67039808544328,
-20.631964120263692,
0
],
[
116.7248183032507,
-20.62054602439853,
0
],
[
116.73651524863865,
-20.667885951800336,
0
],
[
116.68105920150816,
-20.67952294953898,
0
]
]
],
"type": "Polygon"
},
"properties": {
"acquisition_end_utc": "2020-10-15T07:35:35.075306",
"acquisition_start_utc": "2020-10-15T07:35:33.255112",
"coord_center": "-20.650032972483114, 116.70295753757624",
"coord_first_far": "-20.667885951800336, 116.73651524863863",
"coord_first_near": "-20.67952294953898, 116.68105920150815",
"coord_last_far": "-20.62054602439853, 116.7248183032507",
"coord_last_near": "-20.631964120263692, 116.67039808544328",
"fill-opacity": 0,
"incidence_center": "22.76544706618204",
"incidence_far": "23.03044208311177",
"incidence_near": "22.499521201198128",
"license": "Preview images licenced under CC BY-NC 4.0 https://creativecommons.org/licenses/by-nc/4.0/legalcode",
"look_side": "right",
"name": "ICEYE_ARCHIVE_SL_36260_20201015T073533",
"orbit_direction": "ASCENDING",
"polarization": "VV",
"preview_image": "http://iceye-public-images-archive.s3-website.eu-central-1.amazonaws.com/ICEYE_ARCHIVE_SL_36260_20201015T073533.png",
"product_name": "ICEYE_ARCHIVE_SL_36260_20201015T073533",
"product_type": "Spotlight",
"request_data": "Current customers: order via customer success. New customers: https://www.iceye.com/lp/contact-us",
"satellite_look_angle": "20.72",
"stroke": "#ff0000",
"stroke-opacity": 1,
"stroke-width": 2,
"styleHash": "-17b67303",
"styleUrl": "#PolyStyle00",
"timestamp": "2020-10-15T07:35:33.255112"
},
"type": "Feature"
}
I'll extract the URLs and download them.
$ jq '.features[].properties.preview_image' \
ICEYE_Public_Archive_with_Preview_Images_2020.geojson \
> urls.txt
$ cat urls.txt | xargs -P4 -I% wget -c %
The above downloaded ~27 GB of PNGs.
I'll get the GeoFabrik partition IDs and use them to move the images into geographically-named folders.
$ python3
import json
import duckdb
from rich.progress import track
from shapely.geometry import shape
from shapely import wkt
con = duckdb.connect(database='/home/mark/iceye.duckdb')
df = con.sql('INSTALL spatial; LOAD spatial')
filename = 'ICEYE_Public_Archive_with_Preview_Images_2020.geojson'
features = json.loads(open(filename).read())['features']
geofabrik_parts = set()
with open('move_images.sh', 'w') as f:
for feature in track(features):
lat, lon = feature['properties']['coord_center'].split(', ')
lat, lon = float(lat), float(lon)
filename = feature['properties']['preview_image'].split('/')[-1]
df = con.sql("""SELECT *
FROM geofabrik
WHERE ST_CONTAINS(geom, ST_POINT(?, ?))
ORDER BY ST_AREA(geom)
LIMIT 1;""",
params=(lon, lat)).to_df()
if not df.empty:
geofabrik_part = json.loads(df.to_json())['id']['0']\
.replace('/', '_')\
.replace('-', '_')
geofabrik_parts.add(geofabrik_part)
f.write('mv %s %s/\n' % (filename, geofabrik_part))
open('mkdirs.sh', 'w').write('mkdir -p ' + ' '.join(geofabrik_parts))
I'll also rework the rest of the records so that they're easier to import and analyse in DuckDB.
with open('metadata.json', 'w') as f:
for feature in track(features):
lat, lon = feature['properties']['coord_center'].split(', ')
lat, lon = float(lat), float(lon)
filename = feature['properties']['preview_image'].split('/')[-1]
df = con.sql("""SELECT *
FROM geofabrik
WHERE ST_CONTAINS(geom, ST_POINT(?, ?))
ORDER BY ST_AREA(geom)
LIMIT 1;""",
params=(lon, lat)).to_df()
if not df.empty:
geofabrik_part = json.loads(df.to_json())['id']['0']\
.replace('/', '_')\
.replace('-', '_')
# Convert the POLYGON Z to a POLYGON
geom = wkt.loads(wkt.dumps(shape(feature['geometry']),
output_dimension=2))
f.write(json.dumps(
{**feature['properties'],
**{'geom': geom.wkt,
'geofabrik': geofabrik_part,
'filename': geofabrik_part + '/' + filename,
}}) + '\n')
There were 17,301 images that I was able to find a GeoFabrik partition ID for.
$ wc -l metadata.json # 17301
Below is the first metadata record produced by the above script.
$ head -n1 metadata.json | jq -S .
{
"acquisition_end_utc": "2020-10-15T07:35:35.075306",
"acquisition_start_utc": "2020-10-15T07:35:33.255112",
"coord_center": "-20.650032972483114, 116.70295753757624",
"coord_first_far": "-20.667885951800336, 116.73651524863863",
"coord_first_near": "-20.67952294953898, 116.68105920150815",
"coord_last_far": "-20.62054602439853, 116.7248183032507",
"coord_last_near": "-20.631964120263692, 116.67039808544328",
"filename": "australia/ICEYE_ARCHIVE_SL_36260_20201015T073533.png",
"fill-opacity": 0,
"geofabrik": "australia",
"geom": "POLYGON ((116.68105920150816 -20.67952294953898, 116.67039808544328 -20.631964120263692, 116.7248183032507 -20.62054602439853, 116.73651524863865 -20.667885951800336, 116.68105920150816 -20.67952294953898))",
"incidence_center": "22.76544706618204",
"incidence_far": "23.03044208311177",
"incidence_near": "22.499521201198128",
"license": "Preview images licenced under CC BY-NC 4.0 https://creativecommons.org/licenses/by-nc/4.0/legalcode",
"look_side": "right",
"name": "ICEYE_ARCHIVE_SL_36260_20201015T073533",
"orbit_direction": "ASCENDING",
"polarization": "VV",
"preview_image": "http://iceye-public-images-archive.s3-website.eu-central-1.amazonaws.com/ICEYE_ARCHIVE_SL_36260_20201015T073533.png",
"product_name": "ICEYE_ARCHIVE_SL_36260_20201015T073533",
"product_type": "Spotlight",
"request_data": "Current customers: order via customer success. New customers: https://www.iceye.com/lp/contact-us",
"satellite_look_angle": "20.72",
"stroke": "#ff0000",
"stroke-opacity": 1,
"stroke-width": 2,
"styleHash": "-17b67303",
"styleUrl": "#PolyStyle00",
"timestamp": "2020-10-15T07:35:33.255112"
}
I'll create the folders and move the images into them.
$ ./mkdirs.sh
$ ./move_images.sh
Below are the five largest images in this archive.
$ find . -type f -printf '%s %p\n' \
| sort -nr \
| head
2884437 ./kazakhstan/ICEYE_ARCHIVE_SL_20288_20200130T093347.png
2869793 ./malaysia_singapore_brunei/ICEYE_ARCHIVE_SL_25414_20200404T033437.png
2866750 ./madagascar/ICEYE_ARCHIVE_SL_19624_20200121T235943.png
2866636 ./polynesie_francaise/ICEYE_ARCHIVE_SL_19613_20200121T124553.png
2864736 ./china/ICEYE_ARCHIVE_SL_29153_20200521T024752.png
2861626 ./philippines/ICEYE_ARCHIVE_SL_20765_20200207T063343.png
As a test, I ran the 2.75 MB PNG for Kazakhstan through TinyPNG and it was able to reduce it down to 803 KB.
Public Archive Imagery Collection
Below I'll import the metadata into DuckDB and run some analysis on it.
$ ~/duckdb ~/iceye.duckdb
CREATE OR REPLACE TABLE iceye AS
SELECT * EXCLUDE(geom,
timestamp,
acquisition_start_utc,
acquisition_end_utc),
geom::GEOMETRY AS geom,
timestamp::TIMESTAMP as timestamp,
acquisition_start_utc::TIMESTAMP as acquisition_start_utc,
acquisition_end_utc::TIMESTAMP as acquisition_end_utc
FROM READ_JSON('metadata.json');
There are 407 GeoFabrik partitions represented in this dataset.
SELECT geofabrik,
COUNT(*)
FROM iceye
GROUP BY 1
ORDER BY 2 DESC;
┌───────────────────────────┬──────────────┐
│ geofabrik │ count_star() │
│ varchar │ int64 │
├───────────────────────────┼──────────────┤
│ australia │ 847 │
│ china │ 645 │
│ finland │ 439 │
│ gcc_states │ 421 │
│ sudeste │ 323 │
│ socal │ 307 │
│ south_africa │ 284 │
│ mexico │ 278 │
│ norte │ 264 │
│ far_eastern_fed_district │ 256 │
│ nordeste │ 247 │
│ iran │ 232 │
│ chubu │ 225 │
│ new_zealand │ 222 │
│ us_hawaii │ 205 │
│ kyushu │ 192 │
│ kanto │ 190 │
│ congo_democratic_republic │ 184 │
│ us_florida │ 179 │
│ us_texas │ 177 │
│ · │ · │
│ · │ · │
│ · │ · │
│ karlsruhe_regbez │ 1 │
│ montenegro │ 1 │
│ brazil │ 1 │
│ nottinghamshire │ 1 │
│ yukon │ 1 │
│ us_maine │ 1 │
│ lithuania │ 1 │
│ dorset │ 1 │
│ netherlands │ 1 │
│ russia │ 1 │
│ tokelau │ 1 │
│ lorraine │ 1 │
│ nord_pas_de_calais │ 1 │
│ rheinland_pfalz │ 1 │
│ muenster_regbez │ 1 │
│ mittelfranken │ 1 │
│ poitou_charentes │ 1 │
│ britain_and_ireland │ 1 │
│ estonia │ 1 │
│ duesseldorf_regbez │ 1 │
├───────────────────────────┴──────────────┤
│ 407 rows (40 shown) 2 columns │
└──────────────────────────────────────────┘
The imagery spans 16 months. This period covers the first seven satellites ICEYE built and launched.
SELECT STRFTIME(timestamp, '%Y-%m') yyyy_mm,
COUNT(*)
FROM iceye
GROUP BY 1
ORDER BY 1;
┌─────────┬──────────────┐
│ yyyy_mm │ count_star() │
│ varchar │ int64 │
├─────────┼──────────────┤
│ 2019-06 │ 156 │
│ 2019-07 │ 80 │
│ 2019-08 │ 852 │
│ 2019-09 │ 2543 │
│ 2019-10 │ 2175 │
│ 2019-11 │ 1031 │
│ 2019-12 │ 1199 │
│ 2020-01 │ 1500 │
│ 2020-02 │ 1540 │
│ 2020-03 │ 1585 │
│ 2020-04 │ 870 │
│ 2020-05 │ 1243 │
│ 2020-06 │ 902 │
│ 2020-07 │ 592 │
│ 2020-08 │ 453 │
│ 2020-09 │ 454 │
│ 2020-10 │ 126 │
├─────────┴──────────────┤
│ 17 rows 2 columns │
└────────────────────────┘
I'll export the metadata as a GeoPackage (GPKG) file and examine the data in QGIS.
COPY (
SELECT geom,
strftime(timestamp, '%Y-%m') yyyy_mm,
geofabrik,
product_type,
look_side,
orbit_direction,
filename
FROM iceye
)
TO 'iceye_public.gpkg'
WITH (FORMAT GDAL,
DRIVER 'GPKG',
LAYER_CREATION_OPTIONS 'WRITE_BBOX=YES');
Below are the locations of the images broken down by month of capture.
All of their imagery is VV polarised. This is a breakdown of the orbit and look directions and capture modes of the images.
WITH a AS (
select UPPER(CONCAT(orbit_direction, ' ', look_side)) look_dir,
product_type,
COUNT(*) num_images
FROM iceye
GROUP BY 1, 2
)
PIVOT a
ON product_type
USING SUM(num_images)
GROUP BY look_dir
ORDER BY look_dir;
┌──────────────────┬───────────┬───────────────┬──────────┬──────────────┐
│ look_dir │ Spotlight │ SpotlightHigh │ Stripmap │ StripmapHigh │
│ varchar │ int128 │ int128 │ int128 │ int128 │
├──────────────────┼───────────┼───────────────┼──────────┼──────────────┤
│ ASCENDING LEFT │ 1072 │ 67 │ 3116 │ 28 │
│ ASCENDING RIGHT │ 949 │ 71 │ 2961 │ 17 │
│ DESCENDING LEFT │ 1086 │ 75 │ 3382 │ 25 │
│ DESCENDING RIGHT │ 1124 │ 62 │ 3240 │ 26 │
└──────────────────┴───────────┴───────────────┴──────────┴──────────────┘
Below are the locations of the images broken down by imaging mode.
Below are the locations of the images broken down by the look side.
Below are the locations of the images broken down by the direction of orbit.
Below shows the heavy density of imagery over Japan and its neighbouring countries.
Below is a breakdown of the duration of capture.
SELECT ROUND(DATE_PART('milliseconds',
acquisition_end_utc -
acquisition_start_utc) / 1000) AS seconds,
COUNT(*)
FROM iceye
GROUP BY 1
ORDER BY 1;
┌─────────┬──────────────┐
│ seconds │ count_star() │
│ double │ int64 │
├─────────┼──────────────┤
│ 1.0 │ 2 │
│ 2.0 │ 4220 │
│ 3.0 │ 17 │
│ 4.0 │ 39 │
│ 5.0 │ 227 │
│ 6.0 │ 245 │
│ 7.0 │ 50 │
│ 8.0 │ 22 │
│ 9.0 │ 36 │
│ 10.0 │ 9200 │
│ 11.0 │ 3184 │
│ 12.0 │ 38 │
│ 13.0 │ 21 │
├─────────┴──────────────┤
│ 13 rows 2 columns │
└────────────────────────┘
ICEYE's Doha Airport Imagery
I'll walk through the layout of Hamad International Airport in Doha, Qatar (DOH). I used GeoFabrik's OpenStreetMap (OSM) distributable for the GCC and osm_split to extract the geometry and metadata in the following images.
$ mkdir -p ~/osm
$ cd ~/osm
$ wget https://download.geofabrik.de/asia/gcc-states-latest.osm.pbf
$ python ~/osm_split/main.py \
gcc-states-latest.osm.pbf \
--only-h3=86536bc0fffffff,86536bce7ffffff,86536bc57ffffff,86536bcefffffff
Below are the largest resulting GPKG files. Because they're named and organised into folders, it's easier to find which features are of greatest interest.
$ find . -type f -size +140000c -name '*.gpkg' \
| sed 's/\.\///g' \
| tree --fromfile .
.
├── lines
│ ├── aeroway.gpkg
│ ├── barrier.gpkg
│ ├── building
│ │ ├── carport.gpkg
│ │ └── residential.gpkg
│ ├── building.gpkg
│ ├── grass.gpkg
│ ├── highway
│ │ ├── footway.gpkg
│ │ ├── motorway_link.gpkg
│ │ ├── primary.gpkg
│ │ ├── residential.gpkg
│ │ ├── secondary.gpkg
│ │ ├── service.gpkg
│ │ ├── tertiary.gpkg
│ │ ├── trunk.gpkg
│ │ ├── trunk_link.gpkg
│ │ └── unclassified.gpkg
│ ├── parking.gpkg
│ └── sand.gpkg
├── multilinestrings
│ └── public_transport.gpkg
├── other_relations
│ └── restriction.gpkg
└── points
├── barrier.gpkg
└── highway
└── crossing.gpkg
The metadata also is cleaned up by osm_split. Below are a few of the aeroway records.
$ ~/duckdb
.maxrows 15
SELECT aeroway,
ref,
ST_CENTROID(geom) geom
FROM ST_READ('lines/aeroway.gpkg')
WHERE ref IS NOT NULL;
┌──────────────────┬─────────┬───────────────────────────────────────────────┐
│ aeroway │ ref │ geom │
│ varchar │ varchar │ geometry │
├──────────────────┼─────────┼───────────────────────────────────────────────┤
│ runway │ 15/33 │ POINT (51.56649582530363 25.257892352141756) │
│ taxiway │ D │ POINT (51.565072093388345 25.256798269155233) │
│ taxiway │ S │ POINT (51.56108356281074 25.27498354135743) │
│ taxiway │ R │ POINT (51.56107502104281 25.27407744441384) │
│ taxiway │ Q │ POINT (51.56200097774817 25.271494692546064) │
│ taxiway │ B │ POINT (51.56161905243916 25.26717485612459) │
│ taxiway │ P │ POINT (51.56783714510424 25.26356224209176) │
│ taxiway │ L2 │ POINT (51.60339638688445 25.260475582142806) │
│ · │ · │ · │
│ · │ · │ · │
│ · │ · │ · │
│ parking_position │ C3 │ POINT (51.569327 25.266125000000002) │
│ parking_position │ C2 │ POINT (51.568951999999996 25.265988999999998) │
│ parking_position │ C1 │ POINT (51.5685845 25.265855000000002) │
│ parking_position │ C31 │ POINT (51.5682075 25.265717000000002) │
│ parking_position │ C32 │ POINT (51.567836 25.2655805) │
│ parking_position │ C33 │ POINT (51.567325 25.265396000000003) │
│ parking_position │ C34 │ POINT (51.5667935 25.2651995) │
├──────────────────┴─────────┴───────────────────────────────────────────────┤
│ 293 rows (15 shown) 3 columns │
└────────────────────────────────────────────────────────────────────────────┘
The image I'm working with is:
ICEYE_X20_GRD_SLEDF_4049621_20240429T222841.tif
The filename indicates it was captured by ICEYE-X20 on what I suspect is 2024-04-29 22:28:41 UTC. If it is in UTC then the image was captured when it was 01:28 AM on April 30th in Qatar.
I checked tomorrow's departures at DOH and there are planes taking off at all hours of the day, including the middle of the night. The only real gap between departures I saw was between 4:15AM and 7:15AM. This means there should be plenty of planes moving around in the image.
Below I've annotated ICEYE's SAR image with OSM data. I've tinted the image red to help contrast it with everything else. This map has been rotated right 21.5 degrees. The SAR image is 5x5 KM and doesn't cover the entire Airport.
Zooming in, I can see a few Aircraft parked.
If I load the CSI version of this imagery, the aircraft appear as bright rainbows. The model I'll be using hasn't been trained on this sort of imagery so I'll stick to the above single-band image for this exercise.
For context, this is the entire CSI image ICEYE produced.
Downloading SARDet_100K Weights
SARDet_100K hosts their model weights on Baidu Disk and OneDrive. The ZIP file with the weights is 20 GB. Below are its contents.
$ du -hs /home/mark/ckpts/* \
| grep '[0-9][MG]' \
| cut -d/ -f1,5 \
| sed 's/\///g'
1.1G convnext_b_sar
1.1G convnext_b_sar_wavelet
598M convnext_s_sar
603M convnext_s_sar_wavelet
339M convnext_t_sar
344M convnext_t_sar_wavelet
457M fg_frcnn_dota_pretrain_sar_convnext_b_wavelet
228M fg_frcnn_dota_pretrain_sar_convnext_t_wavelet
287M fg_frcnn_dota_pretrain_sar_r101_wavelet
346M fg_frcnn_dota_pretrain_sar_r152_wavelet
308M fg_frcnn_dota_pretrain_sar_swin_s_wavelet
227M fg_frcnn_dota_pretrain_sar_swin_t_wavelet
222M fg_frcnn_dota_pretrain_sar_van_b_wavelet
173M fg_frcnn_dota_pretrain_sar_van_s_wavelet
135M fg_frcnn_dota_pretrain_sar_van_t_wavelet
214M fg_frcnn_dota_pretrain_sar_wavelet_r50
2.2G gfl_r50_denodet_sardet
210M hrsid_frcnn_van_sar_wavelet_bs32_3
481M pretrain_frcnn_dota_convnext_b_sar_wavelet
335M pretrain_frcnn_dota_convnext_s_sar_wavelet
2.3G pretrain_frcnn_dota_r101_sar
312M pretrain_frcnn_dota_r101_sar_wavelet
372M pretrain_frcnn_dota_r152_sar_wavelet
239M pretrain_frcnn_dota_r50_sar_wavelet
333M pretrain_frcnn_dota_swin_s_sar_wavelet
251M pretrain_frcnn_dota_swin_t_sar_wavelet
246M pretrain_frcnn_dota_van_b_sar_wavelet
197M pretrain_frcnn_dota_van_s_sar_wavelet
160M pretrain_frcnn_dota_van_t_sar_wavelet
377M r101_sar_epoch_100.pth
382M r101_sar_wavelet_epoch_100.pth
502M r152_sar_wavelet
231M r50_sar_epoch_100.pth
237M r50_sar_wavelet_epoch_100.pth
210M ssdd_frcnn_van_sar_wavelet_bs32_3
1.1G swin_b_sar
596M swin_s_sar
601M swin_s_sar_wavelet
351M swin_t_sar
356M swin_t_sar_wavelet
338M van_b_sar_wavelet_epoch_100.pth
173M van_s_sar_epoch_100.pth
179M van_s_sar_wavelet_epoch_100.pth
61M van_t_sar_epoch_100.pth
66M van_t_sar_wavelet_epoch_100.pth
SAR Aircraft Training Data
Three of the ten datasets in SARDet_100K contain aircraft imagery.
Datasets | Objects | Resolution | Band | Polarization | Satellites
---------------|-------------|------------|---------|-----------------|------------------------------
MSAR | A, T, B, S | ≤ 1m | C | HH, HV, VH, VV | HISEA-1
SADD | A | 0.5-3m | X | HH | TerraSAR-X
SAR-AIRcraft | A | 1m | C | Uni-polar | GF-3
AIR_SARShip | S | 1,3m | C | VV | GF-3
HRSID | S | 0.5-3m | C/X | HH, HV, VH, VV | S-1B, TerraSAR-X, TanDEMX
ShipDataset | S | 3-25m | C | HH, VV, VH, HV | S-1, GF-3
SSDD | S | 1-15m | C/X | HH, VV, VH, HV | S-1, RadarSat-2, TerraSAR-X
OGSOD | B, H, T | 3m | C | VV/VH | GF-3
SIVED | C | 0.1,0.3m | Ka,Ku,X | VV/HH | Airborne SAR synthetic slice
The object acronyms are (A)ircraft, (B)ridges, (C)ars, (H)arbours, (S)hips and (T)anks.
The ICEYE imagery in this post uses VV polarisation which some of the MSAR data used as well.
SAR radar beams can switch between horizontal and vertical polarisation both when sending and receiving. These are denoted by two letters. For example: HV would mean horizontal transmission and vertical receiving.
Horizontal polarisation is ideal for flat and low surfaces, like rivers, bridges and power lines. Vertical polarisation is ideal for buildings, rough seas and transmission towers.
The aircraft training imagery was collected using the X band in some cases but the C band in others. The X band is good for urban monitoring and seeing through snow but it can't penetrate vegetation very well. This might make aircraft under a natural canopy harder to detect. The ICEYE imagery in this post was all collected with the X band.
DeonDet
I'll run DeonDet on ICEYE's image of Doha International Airport. Below is a configuration file that ships with MSFA.
$ cat ~/SARDet_100K/MSFA/local_configs/DeonDet/gfl_r50_denodet_sardet.py
_base_ = [
'../../configs/_base_/datasets/SARDet_100k.py',
'../../configs/_base_/schedules/schedule_1x.py', '../../configs/_base_/default_runtime.py'
]
num_classes = 6
model = dict(
type='GFL',
data_preprocessor=dict(
type='DetDataPreprocessor',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
bgr_to_rgb=True,
pad_size_divisor=32),
backbone=dict(
type='ResNet',
depth=50,
num_stages=4,
out_indices=(0, 1, 2, 3),
frozen_stages=1,
norm_cfg=dict(type='BN', requires_grad=True),
norm_eval=True,
style='pytorch',
init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')),
neck=dict(
type='FrequencySpatialFPN',
in_channels=[256, 512, 1024, 2048],
out_channels=256,
start_level=1,
add_extra_convs='on_input',
num_outs=5,
norm_cfg=dict(type='GN', num_groups=32, requires_grad=True)),
bbox_head=dict(
type='GFLHead',
num_classes=num_classes,
in_channels=256,
stacked_convs=4,
feat_channels=256,
anchor_generator=dict(
type='AnchorGenerator',
ratios=[1.0],
octave_base_scale=8,
scales_per_octave=1,
strides=[8, 16, 32, 64, 128]),
loss_cls=dict(
type='QualityFocalLoss',
use_sigmoid=True,
beta=2.0,
loss_weight=1.0),
loss_dfl=dict(type='DistributionFocalLoss', loss_weight=0.25),
reg_max=16,
loss_bbox=dict(type='GIoULoss', loss_weight=2.0)),
# training and testing settings
train_cfg=dict(
assigner=dict(type='ATSSAssigner', topk=9),
allowed_border=-1,
pos_weight=-1,
debug=False),
test_cfg=dict(
nms_pre=1000,
min_bbox_size=0,
score_thr=0.05,
nms=dict(type='nms', iou_threshold=0.6),
max_per_img=100))
backend_args = None
train_pipeline = [
dict(type='LoadImageFromFile', backend_args=backend_args),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='Resize', scale=(1024, 1024), keep_ratio=False),
dict(type='RandomFlip', prob=0.5),
dict(type='PackDetInputs')
]
test_pipeline = [
dict(type='LoadImageFromFile', backend_args=backend_args),
dict(type='Resize', scale=(1024, 1024), keep_ratio=False),
# If you don't have a gt annotation, delete the pipeline
dict(type='LoadAnnotations', with_bbox=True),
dict(
type='PackDetInputs',
meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
'scale_factor'))
]
train_dataloader = dict(
dataset=dict(
pipeline=train_pipeline))
val_dataloader = dict(
dataset=dict(
pipeline=test_pipeline))
test_dataloader = dict(
dataset=dict(
pipeline=test_pipeline))
# find_unused_parameters = True
optim_wrapper = dict(
optimizer=dict(
_delete_=True,
betas=(
0.9,
0.999,
), lr=0.0001, type='AdamW', weight_decay=0.05),
type='OptimWrapper')
Tiling Imagery
ICEYE's original imagery was pretty dark when I first ran through this exercise. I wasn't able to get anything detected in the imagery. I ended up manually adjusting the gamma and contrast in QGIS and re-exporting a render in order to have something bright enough for the model.
Below I'll break up the gamma-adjusted image into 4096x4096-pixel GeoTIFFs.
$ cd ~/Qatar_Dwell_Fine_ICEYE_GRD
$ gdalwarp \
-t_srs "EPSG:4326" \
gamma_adjusted.tif \
warped.tif
$ gdal_retile.py \
-s_srs "EPSG:4326" \
-ps 4096 4096 \
-targetDir ./ \
warped.tif
Note, without the call to gdalwarp, you could get the following exception when tiling ICEYE's imagery.
0ERROR 1: Attempt to create -20000x4096 dataset is illegal,sizes must be larger than zero.
Traceback (most recent call last):
File "/usr/bin/gdal_retile.py", line 11, in <module>
sys.exit(main(sys.argv))
File "/usr/lib/python3/dist-packages/osgeo_utils/gdal_retile.py", line 915, in main
dsCreatedTileIndex = tileImage(g, minfo, ti)
File "/usr/lib/python3/dist-packages/osgeo_utils/gdal_retile.py", line 354, in tileImage
createTile(g, minfo, offsetX, offsetY, width, height, tilename, OGRDS, feature_only)
File "/usr/lib/python3/dist-packages/osgeo_utils/gdal_retile.py", line 521, in createTile
s_fh = minfo.getDataSet(dec.ulx + offsetX * dec.scaleX, dec.uly + offsetY * dec.scaleY + height * dec.scaleY,
File "/usr/lib/python3/dist-packages/osgeo_utils/gdal_retile.py", line 212, in getDataSet
resultDS.SetGeoTransform([minx, self.scaleX, 0, maxy, 0, self.scaleY])
AttributeError: 'NoneType' object has no attribute 'SetGeoTransform'
Also, a geologist at Planet Labs pointed out in one of my other recent posts that without a Digital Elevation Model (DEM), gdalwarp will assume the surface is completely flat and at sea level. This might cause positioning issues with any sort of significant topography.
Below is a screenshot of the tiles.
Inference on Tiles
I'll run DeonDet on each tile and output the results into individual folders. Each folder will include an annotated image as well as a JSON file of the detections.
$ for FILENAME in warped\_*\_*.tif; do
STEM=`echo "$FILENAME" | grep -o '[0-9]\_[0-9]'`
echo $STEM
mkdir -p "out.$STEM"
python3 \
~/SARDet_100K/MSFA/image_demo.py \
"./$FILENAME" \
~/SARDet_100K/MSFA/local_configs/DeonDet/gfl_r50_denodet_sardet.py \
--weights ~/ckpts/gfl_r50_denodet_sardet/epoch_best.pth \
--out-dir "out.$STEM"
done
I wrote a Python script that reads the resulting detections and produces a single GPKG file of them.
$ vi json_to_gpkg.py
import itertools
from glob import glob
import json
import geopandas as gpd
from osgeo import gdal
import pandas as pd
from shapely.geometry import box
import typer
app = typer.Typer(rich_markup_mode='rich')
def get_coords(mx, my, x_min, x_size, y_min, y_size):
px = mx * x_size + x_min
py = my * y_size + y_min
return px, py
def get_scores(tile='4_3'):
(x_min,
x_size,
_,
y_min,
_,
y_size) = gdal.Open('warped_%s.tif' % tile).GetGeoTransform()
try:
recs = json.loads(open('out.%s/preds/warped_%s.json' % (tile, tile)).read())
except FileNotFoundError: # Inference failed for some reason
return []
out = []
for num, score in enumerate(recs['scores']):
x1, y1, x2, y2 = recs['bboxes'][num]
r1 = get_coords(x1, y1, x_min, x_size, y_min, y_size)
r2 = get_coords(x2, y2, x_min, x_size, y_min, y_size)
out.append((score, r1[0], r1[1], r2[0], r2[1]))
return out
@app.command()
def main(out:str):
tiles = [x.split('.')[0].split('warped_')[-1]
for x in glob('warped_*_*.tif')]
res = list(itertools.chain.from_iterable(
list(filter(None,
[get_scores(tile)
for tile in tiles]))))
gdf = gpd.GeoDataFrame(
pd.DataFrame({'score': [x[0] for x in res],
'geom': [box(x[1], x[2], x[3], x[4])
for x in res]}),
crs='4326',
geometry='geom')
gdf.to_file(out, driver="GPKG")
if __name__ == "__main__":
app()
$ python json_to_gpkg.py \
gfl_r50_denodet_sardet.gpkg
I've disregarded the classes of objects being detected for this exercise. I just wanted to see how many aircraft were detected with a decent confidence score.
Below are the confidence score buckets.
$ ~/duckdb
SELECT ROUND(score, 1),
COUNT(*)
FROM ST_READ('gfl_r50_denodet_sardet.gpkg')
GROUP BY 1
ORDER BY 1;
┌─────────────────┬──────────────┐
│ round(score, 1) │ count_star() │
│ double │ int64 │
├─────────────────┼──────────────┤
│ 0.1 │ 589 │
│ 0.2 │ 39 │
│ 0.3 │ 7 │
│ 0.4 │ 2 │
│ 0.5 │ 1 │
│ 0.6 │ 1 │
│ 0.8 │ 1 │
│ 0.9 │ 1 │
└─────────────────┴──────────────┘
I didn't have any geofence around the airport so it's good to see there weren't too many false positives above the 0.2 confidence level in Doha's town centre.
When I first zoomed in to the aircraft parking spots by the runway to the right, I thought the model had spotted some parked planes. This turned out not to be the case. Only false positives.
Further Research
As I uncover more insights, I'll update this post with my findings.