AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robot

Jaehwan Jeong^1,2, Tuan-Anh Vu², Mohammad Jony³, Shahab Ahmad³,
Md. Mukhlesur Rahman³, Sangpil Kim^1,†, and M. Khalid Jawed^2,†

¹Korea University, ²University of California, Los Angeles, ³North Dakota State University

1. Project Overview

Duration: July 2 – August 1, 2025
Location: NDSU Experimental Field, Fargo, ND
Objective: Capture time-aligned RGB-D, LiDAR, IMU, and Pose data across three crop sites under realistic outdoor conditions
Focus Sites:
- Site 1: Primary canola site, with repeated daily captures across growth stages and lighting variations
- Site 2: Canola genotype trial with 44 varieties for morphological diversity
- Site 3: Flax trial site with structural variability from differing weed control strategies

2. System Resources

🔧 Hardware Documentation
Robot platform, sensor layout, power system, and networking design for long-term deployment

💻 Software Stack
Control interfaces, real-time streaming modules, and logging mechanisms used during collection

📊 NVS Benchmark
Novel View Synthesis benchmark on AgriChrono across seven scenarios featuring lighting variance and growth span.

💾 AgriChrono Dataset
Public release of RGB-D, LiDAR, IMU, and Pose recordings collected in real-world conditions

3. Field Layout

Main Field Structure

Site 1: Regular Canola (main target crop)
Site 2: Canola Genotype Trial
Site 3: Flax Trial

4. Data Collection Protocol

📆 Collection Phases

Phase	Dates	Frequency	Purpose
Phase 1	July 2–21	4× daily, 7 days/week	Active growth tracking & lighting variation
Phase 2	July 22–Aug 1	2 sessions/week	Slowed growth; less sampling

🧪 Site-wise Collection Frequency

Site	Description	Sessions/Day	Days/Week	Period
Site 1	Main Canola Site	4	7	July 2–21
		4	2	July 22–Aug 1
Site 2	Canola Genotype Trial (Side)	1	1–2 (selected)	July 2–Aug 1
Site 3	Flax Trial (Side)	1	1–2 (selected)	July 2–Aug 1

🛠 Field Conditions

❌ Rainy days: skipped
✅ Wet soil: wooden boards used for UGV traversal
☀️ Lighting Diversity:
- 06:00 (sunrise)
- 11:00 (late morning)
- 16:00 (afternoon)
- 21:00 (sunset)

Sun	Mon	Tue	Wed	Thu	Fri	Sat
		7/1	2	3	4	5
			S1 (4) S2 (1) S3 (1)	S1 (3) S3 (1)	S1 (3)	S1 (2)
6	7	8	9	10	11	12
S1 (4)	S1 (4)	S1 (4)	S1 (4)	S1 (3) S2 (1) S3 (1)	S1 (4)	S1 (4)
13	14	15	16	17	18	19
S1 (4) S2 (1) S3 (1)	S1 (4)	S1 (4)	S1 (4)	S1 (4) S2 (1) S3 (1)	S1 (4)	S1 (4)
20	21	22	23	24	25	26
S1 (4)	S1 (2) S2 (1) S3 (1)	S1 (1)		S1 (1)
27	28	29	30	31	8/1
	S1 (1) S2 (1) S3 (1)	S1 (1)	S1 (1)	S1 (1)	S1 (1) S2 (1) S3 (1)

5. Site Descriptions

🌼 Site 1: Main Canola Site

Dimensions: 50 ft × 3 ft, 4 rows per plot, 9-inch spacing
Planting: June 1, 2025 → Emergence June 7
Variety: InVigor L340PC
Flowering: July 10, 2025
Crop Duration: 90–110 days
Objective: Provide a consistent reference for tracking temporal appearance changes of a single canola variety across growth stages and lighting conditions.

🌼 Site 2: Canola Genotype Trial Site

Design: 11 blocks × 44 genotypes
Size: 44 ft × 3 ft each
Planting: May 30, 2025
Objective: Capture morphological and structural variation by recording diverse genotypes planted in multiple distributed plots.

🌿 Site 3: Flax Trial Site

Design: 4 × 4 plots = 16 plots
Size: 8 ft × 3 ft each
Planting: May 30, 2025
Variety: Gold ND
Duration: 90–120 days
Weed Control:
- 3 blocks herbicide-sprayed by robot
- 1 block hand-weeded
Objective: Introduce complementary structural diversity through a different crop type and plot arrangement, including weed control treatments for comparative analysis.

6. Data Structure

Raw data format (`raw_data/[site]/[timestamp]/`)

[timestamp]/
├── LiDAR/
│   ├── imu_sync.bin              ← Raw IMU data from Mid-360 LiDAR
│   ├── pointcloud_sync.bin       ← Timestamped LiDAR point clouds (binary)
├── RGB-D/
│   ├── L.svo2                    ← Left ZED X SVO recording
│   ├── L_info.csv                ← Left ZED IMU and timestamp info
│   ├── R.svo2                    ← Right ZED X SVO recording
│   ├── R_info.csv                ← Right ZED IMU and timestamp info
├── sync_time.txt                 ← Global time sync info (ZED ↔ LiDAR)

Extracted data format (`extracted_data/[site]/`)

[site]/
├── [timestamp]_RGB.mp4           ← 4 RGB views: L_L, L_R, R_L, R_R
├── [timestamp]_Depth.mp4         ← RGB + Depth views: L_L_RGB, R_L_RGB, L_L_Depth, R_L_Depth
├── [timestamp]_Lidar.mp4         ← RGB + Depth + LiDAR point clouds
├── [timestamp].tar.gz            ← Compressed archive of the extracted [timestamp]/ folder

Extracted session folder (`extracted_data/[site]/[timestamp]/`)

[timestamp]/
├── depth_npz_L/                  ← Depth (.npz) aligned to L_L camera
│   ├── 00000.npz
├── depth_npz_R/                  ← Depth (.npz) aligned to R_L camera
│   ├── 00000.npz
├── depth_png_L/                  ← Depth visualization (.png), aligned to L_L
│   ├── 00000.png
├── depth_png_R/                  ← Depth visualization (.png), aligned to R_L
│   ├── 00000.png
├── frame_L/                      ← PNG RGB frames from left ZED X
│   ├── L_00000.png               ← L_L: left sensor of left ZED X
│   ├── R_00000.png               ← L_R: right sensor of left ZED X
├── frame_R/                      ← PNG RGB frames from right ZED X
│   ├── L_00000.png               ← R_L: right sensor of left ZED X
│   ├── R_00000.png               ← R_R: right sensor of left ZED X
├── lidar/
│   ├── fov150                    ← LiDAR point clouds cropped to match ZED X stereo FoV (~150°)
│   │   ├── 00000.ply             ← LiDAR point cloud for frame 0
│   ├── fov360                    ← Full-range LiDAR point clouds (raw 360° FoV)
│   │   ├── 00000.ply             
│   ├── lidar_info.csv            ← Per-frame timestamp and IMU data from LiDAR
├── zed_info.csv                  ← Per-frame timestamp and IMU data from ZED L and R
├── pose_info.csv                 ← Per-frame timestamp and VIO Pose from ZED L and R

CSV Format

`zed_info.csv`

Column	Description
`L_frame_id`, `R_frame_id`	Matched frame IDs from left/right ZED
`L_timestamp`, `R_timestamp`	Absolute timestamps (in seconds)
`relative_time`	Time elapsed from the first ZED frame (starts at 0.0)
`L_`, `R_`	IMU data from each ZED (accel_x/y/z, gyro_x/y/z)

`pose_info.csv`

Column	Description
`L_frame_id`, `R_frame_id`	Matched frame IDs from left/right ZED
`L_timestamp`, `R_timestamp`	Absolute timestamps (in seconds)
`L_`, `R_`	VIO Pose data from each ZED (trans_x/y/z, orien_x/y/z/w)

`lidar_info.csv`

Column	Description
`frame_id`	Frame index aligned with ZED relative time
`timestamp`	Absolute timestamps (in seconds)
`accel_`, `gyro_`	IMU data from LiDAR at the corresponding time

7. Camera Calibration

📷 Left ZED X (4.0 mm)

Resolution: 1920 × 1080 (FHD)
Intrinsic (shared by both sensors):

fx = 1258.97, fy = 1258.97
cx =  916.48, cy =  553.83

Extrinsic (Right → Left transform):

[ 1.000000  0.000000  0.000000  120.009026 ]
[ 0.000000  1.000000  0.000000    0.000000 ]
[ 0.000000  0.000000  1.000000    0.000000 ]
[ 0.000000  0.000000  0.000000    1.000000 ]

📷 Right ZED X (4.0 mm)

Resolution: 1920 × 1080 (FHD)
Intrinsic (shared by both sensors):

fx = 1261.83, fy = 1261.83
cx =  978.86, cy =  535.06

Extrinsic (Right → Left transform):

[ 1.000000  0.000000  0.000000  120.212349 ]
[ 0.000000  1.000000  0.000000    0.000000 ]
[ 0.000000  0.000000  1.000000    0.000000 ]
[ 0.000000  0.000000  0.000000    1.000000 ]

📜 Citation

@article{jeong2025agrichrono,
  title={AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robot},
  author={Jeong, Jaehwan and Vu, Tuan-Anh and Jony, Mohammad and Ahmad, Shahab and Rahman, Md. Mukhlesur and Kim, Sangpil and Jawed, M. Khalid},
  journal={arXiv preprint arXiv:2508.18694},
  year={2025},
}

Name		Name	Last commit message	Last commit date
Latest commit History 85 Commits
assets		assets
benchmark		benchmark
datasets		datasets
hardware		hardware
software		software
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robot

1. Project Overview

2. System Resources

3. Field Layout

4. Data Collection Protocol

📆 Collection Phases

🧪 Site-wise Collection Frequency

🛠 Field Conditions

5. Site Descriptions

🌼 Site 1: Main Canola Site

🌼 Site 2: Canola Genotype Trial Site

🌿 Site 3: Flax Trial Site

6. Data Structure

Raw data format (`raw_data/[site]/[timestamp]/`)

Extracted data format (`extracted_data/[site]/`)

Extracted session folder (`extracted_data/[site]/[timestamp]/`)

CSV Format

`zed_info.csv`

`pose_info.csv`

`lidar_info.csv`

7. Camera Calibration

📷 Left ZED X (4.0 mm)

📷 Right ZED X (4.0 mm)

📜 Citation

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robot

1. Project Overview

2. System Resources

3. Field Layout

4. Data Collection Protocol

📆 Collection Phases

🧪 Site-wise Collection Frequency

🛠 Field Conditions

5. Site Descriptions

🌼 Site 1: Main Canola Site

🌼 Site 2: Canola Genotype Trial Site

🌿 Site 3: Flax Trial Site

6. Data Structure

Raw data format (raw_data/[site]/[timestamp]/)

Extracted data format (extracted_data/[site]/)

Extracted session folder (extracted_data/[site]/[timestamp]/)

CSV Format

zed_info.csv

pose_info.csv

lidar_info.csv

7. Camera Calibration

📷 Left ZED X (4.0 mm)

📷 Right ZED X (4.0 mm)

📜 Citation

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Raw data format (`raw_data/[site]/[timestamp]/`)

Extracted data format (`extracted_data/[site]/`)

Extracted session folder (`extracted_data/[site]/[timestamp]/`)

`zed_info.csv`

`pose_info.csv`

`lidar_info.csv`

Packages