All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Automatic
pc_alignstep in the Earth altimetry block, gated by a new--pc_alignCLI flag (defaultTrue; disabled automatically when--plot_altimetry/--plot_icesatisFalse). Runspc_alignagainst ICESat-2 ATL06-SR, evaluates whether the aligned DEM is worth keeping, and appends the outcome as one or more report pages:- Always: an alignment report page with the parameters table, a single-row horizontal stats table (p16/p50/p84 beg/end, north/east/down shifts, translation magnitude, values to 2 sig figs), a description explaining what
pc_aligndoes and the meaning of every column in the tables above, and a bold status line for the outcome of this run. - On success (p50 drops toward 0 by more than
improvement_threshold_pct, default 5%, andpc_alignactually wrote an aligned DEM): three additional full-page diagnostic figures against the aligned DEM — a pre-/post-alignment landcover histogram, the full profile, and the best/worst 1 km segments. - On insufficient ATL06-SR coverage or no meaningful improvement: the aligned DEM on disk is cleaned up so its presence is a truthy signal that the alignment is worth using.
- Always: an alignment report page with the parameters table, a single-row horizontal stats table (p16/p50/p84 beg/end, north/east/down shifts, translation magnitude, values to 2 sig figs), a description explaining what
Altimetry.align_and_evaluate(...)(new method) returning a plainAlignmentResultdataclass (status ∈ {"insufficient_points", "no_improvement", "success"},alignment_report_df,aligned_dem_fn,improvement_pct,message,parameters_used). Does not import anyfpdf/ report dependencies, so it is safe to call from notebooks.plot_alignedkwargs onAltimetry.histogram_by_landcoverandAltimetry.plot_best_worst_segments:histogram_by_landcover(plot_aligned=True)overlays the pre- and post-alignment distributions using shared bin edges and renders two vertically stacked per-landcover stats text boxes whose outline colors match the bar colors (color = legend).plot_best_worst_segments(plot_aligned=True)keeps segment selection fixed (based on the unaligneddhso segments are comparable), overlays aligned DEM heights on each segment, and appends aligned Median/NMAD to the segment titles.
AlignmentReportPagedataclass inasp_plot.report: a report-section type that renders a kwargs table + single-row stats table + description + bold status line + optional figure with caption. Body text blocks render left-aligned to avoid justified word-spacing gaps.
- Processing Parameters is now page 2 of the PDF, immediately after the DEM Summary on the title page, instead of the trailing appendix. Page order is now: title + DEM summary → processing parameters → diagnostic figures → (if any) alignment results.
plot_atl06sr_dem_profile(plot_aligned=True): the lowerdhpanel now plots the post-alignment residuals (icesat_minus_aligned_dem) with Med/NMAD recomputed against the aligned DEM, with the legend entry tagged"(Aligned DEM)". The upper elevation panel still overlays both the unaligned and aligned DEM for comparison.plot_aligned=Falsebehavior is unchanged.
- Report panel order: Disparity maps now follow the Bundle Adjust panels, and DEM Results precedes Detailed Hillshade (was: Hillshade → DEM Results → Disparity).
- Input Scenes caption clarifies that mapprojected scenes are RPC-orthorectified against a reference DEM to roughly pre-align the stereo pair prior to correlation (reducing disparity search range), which addresses confusion for readers coming from non-ASP photogrammetry workflows.
- Match Points caption notes these come from
stereo_corr's initial interest point matching step (used to set search windows), not dense correlation.
- Acquisition Date(s) row on the DEM Summary title-page table, populated when recoverable from scene metadata.
get_acquisition_dates()helper inutils.py: readsFIRSTLINETIMEfrom WorldView/Maxar XMLs and parses the capture timestamp fromAST_L1A_...file/directory names. Deduplicates and sorts; returns an empty list if no date can be found, in which case the summary-table row is omitted.- Unit tests for
get_acquisition_dates()covering WorldView XMLs, ASTER filenames (top-level and in subdirectories), dedupe, and sorting of multi-date pairs.
--report_filenameaccepts absolute and relative paths (not just bare filenames), and~in CLI path arguments is expanded (#113).
- ICESat-2 time filter default changed to
"all"(full mission range) instead of auto-detect ±1 year. The--atl06sr_time_rangeCLI option now accepts"all"(default),"auto"(XML metadata ±time_buffer_days),"START,END", or a single date (buffered). Programmatic API:_resolve_time_range()andrequest_atl06sr_multi_processing()take a newtime_rangeparameter ("all"or"buffered") with cascade:t0/t1>scene_date> XML metadata > fall back to"all".t1is truncated to midnight UTC for stable parquet caching. - ESA WorldCover sampled locally from AWS S3 COGs via
rasteriovsicurl instead of through the slow SlideRulesamplesparameter. WorldCover is now sampled insiderequest_atl06sr_multi_processingbefore the parquet save, so the column is persisted in the cache and doesn't have to be re-sampled on subsequent runs. COP30 also sampled alongside (asset name corrected toesa-copernicus-30meter). - 3σ outlier filter applied by default in
atl06sr_to_dem_dh(andplanetary_to_dem_dh) using the true mean ± 3·standard deviation (not NMAD). Passn_sigma=Noneto skip. Dh colorbars and histograms use symmetric ±|filtered min/max| (≈ ±3σ) centered on 0 as display limits; all data is still plotted. Displayed stats remain Median / NMAD. - Profile plot (
plot_atl06sr_dem_profile) restructured: stacked elevation/dh plots on the left (shared x-axis, no vertical gap, grid lines), map view on the right spanning both rows. Figure reshaped to 16×8. Dh points colored gray instead of salmon. - Best/worst segments (
plot_best_worst_segments) simplified to a 1×2 figure (removed the context map). Scoring formula changed from|median(dh)| + NMAD(dh)to3·|median(dh)| + NMAD(dh)so a large median bias can't be hidden by a small NMAD. Labels "Better agreement" / "Worse agreement" instead of "Best" / "Worst". CLI report caption documents the formula. - Parquet cache saved next to the ASP processing directory (
self.directory) instead of the current working directory. - SlideRule logging silenced (
verbose=False, WARNING level, explicit filter onsliderule.session).
filter_outliers()method: removes dh points beyondn_sigma× standard deviation from the mean.sample_esa_worldcover()method: samples ESA WorldCover 10m values from AWS S3 COGs for manually-loaded data (auto-called insiderequest_atl06sr_multi_processing).plot_best_worst_segments()method: 1×2 figure showing 1 km segments with better and worse ICESat-2 vs DEM agreement.- ICESat-2 time filtering documentation section in
docs/cli/asp_plot.mdexplaining the three modes.
- Parquet cache regeneration bug: SlideRule mutates the
parmsdict by injecting a random temp file path atoutput.pathduringrun(), causing the string comparison to fail on every subsequent run.outputis now stripped from both sides of the comparison and from stored parameters. - Parquet cache error swallowing: the broad
try/exceptaround the cache comparison also wrapped the SlideRule API call; narrowed it so API errors propagate instead of being silently eaten. - Histograms no longer cut data: replaced
range=(which excludes data outside the range from the bins) withax.set_xlim(), so all data is plotted and used in stats. - Single-date CLI argument now uses
scene_datebuffering instead of being treated as a start date. - COP30 SlideRule asset name corrected (
esa-copernicus-30meter, notcop30-dem).
- Pinned
sliderule>=5.3.0to pick up temp file handling fixes.
- Asymmetry angle calculation: ECEF ground point z-coordinate was incorrectly set to 0 (equatorial plane) instead of using the proper WGS84 ellipsoid position from pyproj, producing wrong values at non-equatorial latitudes
- Stereo geometry functions (
get_convergence_angle,get_bh_ratio,get_bie_angle,get_asymmetry_angle) extracted to module-level instereopair_metadata_parser.pyfor reuse and testability
- Unit tests for convergence angle, B/H ratio, BIE, and asymmetry angle calculations, including a regression test for the ECEF z=0 bug
- New
--atl06sr_time_rangeCLI option for controlling ICESat-2 ATL06-SR time filtering: use"all"for full mission range, or"START,END"for a custom date range (e.g."2020-01-01,2024-12-31") - Corresponding
t0/t1parameters onAltimetry.request_atl06sr_multi_processing()andAltimetry._resolve_time_range()for programmatic use - New WorldView-3 UCSD example notebook (
worldview_spacenet_ucsd_stereo.ipynb) using publicly available IARPA CORE3D data, with comprehensive stereopair selection analysis - Example report:
WorldView_UCSD-asp-plot-report.pdf
Alignment.pc_align_report()andAlignment.apply_dem_translation()now returnNonegracefully when pc_align log files are not found, instead of crashing withTypeErrorAltimetry.alignment_report()handles missing pc_align results with a warning instead of crashingkey_for_aligned_demparameter inAltimetry.alignment_report()now defaults to theprocessing_levelvalue instead of being hardcoded to"ground"
- Planetary altimetry validation: LOLA (Moon) and MOLA (Mars) DEM comparison via the ODE Granular Data System (GDS) REST API, analogous to the existing ICESat-2 workflow for Earth DEMs
- New
request_planetary_altimetryCLI tool to submit async LOLA/MOLA data requests with email notification, saving request metadata toaltimetry_request_info.yml - New
--plot_altimetryflag on theasp_plotCLI with automatic body detection (Earth → ICESat-2, Moon → LOLA, Mars → MOLA) - New
--altimetry_csvflag to pass a pre-downloaded LOLA/MOLA*_topo_csv.csvfile for planetary altimetry plots detect_planetary_body()utility function: detects Earth/Moon/Mars from DEM CRS WKTget_planetary_bounds()utility function: converts DEM bounds to planetocentric 0-360 lon/lat for GDS queriesAltimetry.load_planetary_csv(): loads LOLA or MOLA CSV with column validation and helpful error messagesAltimetry.planetary_to_dem_dh(): computes altimetry-minus-DEM differences using WKT-based CRS (supports planetary DEMs without EPSG codes)Altimetry.mapview_plot_planetary_to_dem(): DEM hillshade with dh point overlayAltimetry.histogram_planetary_to_dem(): dh histogram with n/median/NMAD statistics- Lazy SlideRule initialization:
Altimetry.__init__no longer requires an internet connection; SlideRule is initialized on first ICESat-2 method call - LOLA/MOLA altimetry sections added to LRO NAC, Mars MGS MOC, Mars MGS MOC NA, and Mars MRO HiRISE example notebooks
- Unit tests for body detection, planetary bounds, lazy init, CSV loading/validation, and planetary dh computation
--plot_icesatis now a deprecated alias for--plot_altimetry(prints deprecation warning if used)- Basemaps are automatically skipped for non-Earth DEMs
pyyamladded as an explicit dependency
- Match points now overlay on non-mapprojected images using alignment transform matrices (
run-align-{L,R}.txt), replacing the previous blank-right-panel behavior - Report command string recorded in PDF report via new
report_commandparameter incompile_report() - Pixel-unit scalebar for non-mapprojected disparity plots (mapprojected scenes continue to use GSD-based scalebar)
- Guard with
FileNotFoundErrorwhen alignment matrix files are missing for non-mapprojected match point overlay - Warning when
unit="meters"is passed for non-mapprojected disparity (unsupported, falls back to pixels) - Test coverage for non-mapprojected stereo code paths (9 new tests with resampled ASTER test data)
- Report figures are now fitted to page dimensions, preventing overflow and cutoff for large/wide figures
- Report caption reserve is now dynamically calculated from actual caption length instead of a hardcoded 20mm
- Input Scenes caption updated to explain alignment rotation applied to non-mapprojected imagery
- Match points right subplot title simplified from "Right (scenes shown only if mapprojected)" to "Right"
save_figure()default DPI changed from hardcoded 150 toNone(uses figure's own creation DPI), fixing pixelated ICESat-2 report figures- ICESat-2 altimetry figures created at 220 DPI for high-quality PDF embedding
- CLI parameter values are now quoted with
shlex.quote()for proper reconstruction of commands with spaces - Cleaned up example notebook report links and removed stale PDF files
- Removed unnecessary
read_align_matrix()method; alignment matrices are loaded inline vianp.loadtxt()
- Disparity plot scale for non-mapprojected scenes: GSD-based rescale was producing near-zero values from the identity transform; now skips rescaling and uses pixel-unit scalebar instead
- Match point plot whitespace for non-mapprojected scenes caused by a 1x1 dummy image plotted underneath scatter points
- Pixelated ICESat-2 ATL06-SR figures in PDF reports caused by
save_figure()overriding figure DPI with 150
- New
_select_best_track()method to find the RGT/cycle/spot combination with the most valid ATL06-SR points for profile plotting - New
histogram_by_landcover()method producing a histogram of ICESat-2 vs DEM differences with per-landcover-class statistics (count, median, NMAD) using ESA WorldCover - New
plot_atl06sr_dem_profile()method with a three-row figure: combined elevation + dh profile with dual y-axes, two 1 km zoom segments (best/worst agreement scored by |median(dh)| + NMAD), and DEM hillshade map with track overlay - Server-side time filtering for SlideRule API requests via new
_resolve_time_range()method with three-tier cascade: explicitscene_dateparameter, auto-detect from stereopair XML metadata, or 2-year fallback scene_dateandtime_buffer_daysparameters added torequest_atl06sr_multi_processing()- Module-level
ICESAT2_MISSION_STARTconstant andWORLDCOVER_NAMESdictionary for reuse across methods - Module-level
_nmad()helper function (Normalized Median Absolute Deviation) - Time range labels displayed on ICESat-2 plot titles
- Tests for
_select_best_track,histogram_by_landcover,plot_atl06sr_dem_profile, and_resolve_time_range
- Migrated SlideRule API from legacy
icesat2.atl06p()to x-seriessliderule_api.run("atl03x")with automatic index and column normalization - Simplified ICESat-2 report section: single
"all"processing level with landcover histogram and profile plot, replacing the previous multi-level (all + ground) workflow with temporal filtering and plain histograms - Report section ordering: bundle adjustment plots now appear after match points and before DEM hillshade
- Profile plot legend now includes axis labels (left/right) for all entries and embeds Med/NMAD statistics in the dh legend item
--icesat_filter_dateCLI option (time filtering is now automatic via_resolve_time_range())- Commented-out
plot_atl06sr_dem_profiles()stub and ATL03 request stub (replaced by implemented methods) - Duplicated WorldCover classification table from
filter_esa_worldcover()docstring (now referencesWORLDCOVER_NAMES)
TypeError: Cannot subtract tz-naive and tz-aware datetime-like objectsinpredefined_temporal_filter_atl06srwhen scene date is UTC-aware but DataFrame index is tz-naiveKeyError: 'translation_magnitude'inalignment_report()when requested processing level has no data (now returns early with a warning)TypeError: unhashable type: 'numpy.ndarray'inhistogram_by_landcovercaused by parquet round-trip deserializing arrays as Python listsTypeError: 'int' object is not callablewhen builtinlen()was shadowed by thelen=40parameter insiderequest_atl06sr_multi_processingOverflowError: cannot convert float infinity to integerin profile segment selection when median point spacing is zero
- ASP version and asp_plot version displayed on report title page
- Copyright overlay ("© Vantor {year}") on WorldView satellite imagery in scene, match point, and detailed hillshade plots
detect_vantor_satellite()utility to identify WorldView imagery from XML SATID tagsadd_copyright_overlay()utility for matplotlib axesProcessingParameters.get_asp_version()method to extract ASP version from log filesRaster._mask_nodata()private helper to consolidate nodata/invalid value maskingRaster._load_and_diff_rasters_da()private static method returning xarray DataArray for raster differencing
Raster.get_bounds()now usesself.ds.bounds(rasterio) instead of opening a redundant rioxarray datasetRaster.compute_difference()usesrio.to_raster()for saving whensave=True, avoiding manual profile constructionStereoPlotter.plot_detailed_hillshade()reuses existingraster.ds.transforminstead of reopening the DEM file- Consolidated duplicated nodata masking logic into
Raster._mask_nodata() - Updated ASTER and WorldView example notebooks
- Updated README installation instructions with conda-forge as recommended install method
- Updated README release process documentation for automated pipeline
- Added missing runtime dependencies to
pyproject.toml:geopandas,matplotlib-scalebar,sliderule
- Automated PyPI publishing via OIDC trusted publishing on GitHub Release
- conda-forge reference recipe for staged-recipes submission
- Runtime dependencies declared in
pyproject.toml(pip install asp-plotnow installs all deps)
- Replaced deprecated
actions/create-release@v1withsoftprops/action-gh-release@v2in release workflow - Added missing dependencies to
environment.yml:pyproj,scipy,shapely,xarray
- Structured PDF report generation with title page, section headings, figure captions, DEM metadata summary table, and runtime summary table
- New
report.pymodule containingReportSectionandReportMetadatadataclasses,ASPReportPDFclass, andcompile_report()function - DEM metadata (dimensions, GSD, CRS, nodata %, elevation range) automatically collected and displayed on the report title page
- Figure captions describing each plot in the generated PDF report
- Page headers (report title) and footers (page numbers) throughout the report
- Tests for report dataclasses and PDF compilation (8 new tests)
- Replaced
markdown-pdfdependency withfpdf2(available on conda-forge, enabling conda-only installation) - Reordered report sections: Input Scenes and Stereo Geometry now appear before DEM results, matching the logical processing flow
- Report generation moved from
utils.pyto dedicatedreport.pymodule - PNG images are now embedded directly in the PDF (eliminated intermediate PNG-to-JPEG conversion step)
- Dependency on
markdown-pdf(pip-only package that blocked conda-forge packaging)
- Satellite attitude (ATT) parsing from DigitalGlobe/Maxar XML files: new
getAtt()andgetAtt_df()methods onStereopairMetadataParser, mirroring the existing ephemeris parsing - New
satellite_position_orientation_plot()method onStereoGeometryPlotterproducing a 3x2 figure showing position covariance, roll/pitch/yaw orientation, and attitude covariance for each scene - Attitude data (
att_df) now included in catalog ID dictionaries returned byget_catid_dicts()
- Ephemeris covariance columns in
getEphem_gdf()renamed fromx_cov, y_cov, ...tocov_11, cov_12, cov_13, cov_22, cov_23, cov_33for clarity
- New WorldView SpaceNet Atlanta stereo processing example notebook using publicly available data
- New utility function
get_utm_epsg()for determining UTM EPSG code from longitude/latitude - New
Raster.get_utm_epsg_code()method for estimating UTM zone from raster location - New
StereopairMetadataParsermethods:get_pair_utm_epsg(),get_intersection_bounds(),get_scene_bounds()
Alignment.get_alignment_report()now returns North-East-Down shift keys (north_shift,east_shift,down_shift) instead of ECEF Cartesian keys (x_shift,y_shift,z_shift).- Renamed
worldview_comprehensive.ipynbtoworldview_utqiagvik_stereo.ipynb.
- Fixed
geodiffcommand in bundle adjustment processing: corrected argument order (DEM must come before CSV) and csv-format syntax (spaces instead of commas between column specs) - Fixed graceful handling when
--mapproj-demflag was not used in bundle_adjust: geodiff plots are now skipped with a warning instead of causing the entire bundle adjustment section to fail - Fixed relative reference DEM paths read from log files not being resolved to absolute paths, causing "file not found" errors
- Jitter solved ASTER example processing notebook
- Currently the GSD for bundle adjustment calculations is pulled from the metadata for WorldView scenes. ASTER does not contain this metadata, so a fallback value is used (1 m GSD), which effectively renders the bundle adjustment calculations always in pixels. We will eventually want to support an argument or other parser, but this is not important at the moment and instead this approach gracefully allows plotting to continue without erroring out.
- Several new example processing notebooks in
notebooks/ - A new argument
datetoAltimetry.predefined_temporal_filter_atl06sr, which can be used to pass the capture date of the scene for filtering. Previously, the date was read from metadata, but that only works for WorldView right now. - A new flag to the
asp_plotCLI:--icesat_filter_date, which passes the YYYY-MM-DD formatted date to the icesat filtering method
- Previously, when void pixels were contained in the detailed mapprojected subset images in the detailed hillshade plots, the entire subset plot would appear blank. This is fixed by masking no data values and calculating the color ranges excluding them.
- Similarly, the disparity maps were also improperly showing data void areas. This is fixed by better handling of void areas during the disparity map calculations and plotting.
- New
notebooks/Mars_MOCexample
- Added a regular hillshade fallback to
StereoPlotter.plot_detailed_hillshade()for the case where*-IntersectionErr.tifwas not produced and is not available for detailed hillshade plots.
- Extracted common hillshade plotting logic in
StereoPlotterto utility function.
- Support for non-terrestrial (planetary) ASP processing, tested with Lunar Reconnaissance Orbiter (LRO) Narrow Angle Camera (NAC) data
- New
--plot_geometryCLI flag to optionally skip stereo geometry plots (default: True) - New
--subset_kmCLI flag to configure hillshade subset size in kilometers (default: 1.0 km) - Example notebook for LRO NAC processing in
notebooks/LRO_NAC/ - Detection and handling of non-georeferenced (raw, non-map-projected) raster data
- API change:
ScenePlotter.plot_orthos()renamed toScenePlotter.plot_scenes()for sensor-agnostic naming ScenePlotterno longer depends onStereopairMetadataParser, making it compatible with non-Earth sensors- Scene plots now automatically detect and display whether images are map-projected or raw
- Scene plot titles now show filenames instead of Earth-specific metadata (catalog ID, GSD)
Raster.transformproperty now returnsNonefor non-georeferenced images (identity transform) instead of identity Affine- Suppressed
NotGeoreferencedWarningwhen opening non-georeferenced rasters - Match points plot clarification text updated: "scenes shown only if mapprojected"
StereoPlotter.is_mapprojected()method - replaced with simplerRaster.transformcheck
- Simplified map-projection detection logic using
Raster.transform is Nonecheck
- Moved existing example notebooks into
WorldViewsub-directory, since we plan to introduce other sensors and we'd like to keep things separated in our examples.
- While moving and re-running notebooks, it was noted that
Altimetry.plot_atl06had a bug whenplot_dem=True. Therasterio.plot.showwas improperly imported. This is properly imported now.
downsampleparameter toRasterclass for efficient downsampled reading- Lazy-loaded
dataproperty onRasterclass using@propertydecorator save_raster()static method for flexible raster saving with reference metadata- Optional
saveparameter (defaultFalse) tocompute_difference()method _calculate_downsampled_shape()private method for modular downsampling logic- Comprehensive test suite for
RasterandColorBarclasses (21 new tests intest_utils.py) - Explicit
rioxarraydependency toenvironment.yml(was previously an implicit dependency via geoutils)
- Refactored
Rasterclass to remove dependency ongeoutils load_and_diff_rasters()now usesrioxarrayfor efficient reprojection and cropping (matching geoutils behavior with simpler implementation)compute_difference()no longer saves by default (usesave=Trueto enable)- Difference rasters are now cropped to the intersection of both input rasters (matching geoutils behavior)
- Updated
altimetry.pyto use native rasterio plotting instead of geoutils
- Dependency on
geoutils(>=0.1.9) - Dependency on
xdem(was unused)
- Extracted downsampling logic into reusable private method
- Added properties for
dataandtransformwith lazy loading - Improved separation of concerns between data loading and file I/O
- Small typo csm_camera CLI help text
- Improper passing of map_crs into csm_camera CLI tool fixed
- Sometimes while trimming linescan cameras to only the rows of image capture in the csm_camera utilities, the indices of first and last collection line are reversed. I think this has to do with ascending versus descending orbits, but I didn't investigate deeply. I did add a conditional check to the responsible function to switch the index slicing in this case.
- Added comprehensive docstrings throughout the codebase for better code documentation
This is the first stable release of asp_plot. While it was previously available as a pre-1.0 package,
this release marks a commitment to proper versioning and documentation.
- New
stereo_geomcommand-line tool for visualizing stereo geometry - Added comprehensive docstrings to CLI tools
- Created this CHANGELOG file for better tracking of changes
- Added support for multiple XML files with automatic mosaicking via
dg_mosaic
- Fixed subprocess handling in
stereopair_metadata_parser.pyfor multiple XML file processing
Combined beta release of asp_plot since version 0.0.1, before a proper change log was established.
- Initial public release of
asp_plot - Support for bundle adjust visualization
- Support for stereo visualization
- Report generation capabilities
- CSM camera plot tool