Get Started in 10 Minutes¶

Altay Sansal

Sep 22, 2026

8 min read

In this page we will be showing basic capabilities of MDIO.

For demonstration purposes, we will ingest a public raw shot record from the Soda Lake 2010 3D/3C seismic survey (Geothermal Data Repository / AWS Open Data, CC-BY-4.0).

We use one ~7.5 MiB field record: F7733R1.SGY. No AWS credentials are required. The file is a 3C common-source gather: shot 7733, 958 traces (1 aux + 3 × 319 receivers), 2000 samples at 2 ms.

Warning

For plotting and remote ingestion the notebook requires Matplotlib and aiohttp as a dependency. Please install it before executing using pip install matplotlib aiohttp or conda install matplotlib aiohttp.

Defining the SEG-Y Dataset¶

Since MDIO 0.8 we can directly ingest remote SEG-Y files. To make the header scan performant we can also set up an environment variable for MDIO. See here for when to use this: Buffered Reads in Ingestion.

The gather is a single live shot, so the live mask is all True. The first trace is an auxiliary channel at (0, 0) and is much louder than the seismic. Plots below drop it and keep one of the three components.

The following environment variables are essential here:

MDIO__IMPORT__CLOUD_NATIVE tells MDIO to do buffered reads for headers due to remote file.
MDIO__IMPORT__SAVE_SEGY_FILE_HEADER makes MDIO save the SEG-Y specific file headers (text, binary) which is not strictly necessary for consumption and is disabled by default.

import os

os.environ["MDIO__IMPORT__CLOUD_NATIVE"] = "true"
os.environ["MDIO__IMPORT__SAVE_SEGY_FILE_HEADER"] = "true"

input_url = "https://gdr-data-lake.s3.us-west-2.amazonaws.com/soda_lake/raw_seismic/2010/v1.0.0/F7733R1.SGY"

Ingesting to MDIO¶

To do this, we can use the convenient SEG-Y to MDIO converter.

The shot identifier is the field record number at byte 9, and the channel number is at byte 13.

MDIO uses TGSAI/segy to parse the SEG-Y; the field names conform to its canonical keys defined in SEGY Binary Header Keys and SEGY Trace Header Keys. Since MDIO v1 we also introduced templates for common seismic data types. For this shot gather we use the StreamerShotGathers2D template here, which expects shot_point and channel.

We will also specify the units for the time domain. The spatial units will be automatically parsed from SEG-Y binary header. However, there may be a case where it is corrupt in the file, for that see the Fixing X/Y Units Issues section.

In summary, we will use the byte locations as defined for ingestion.

import matplotlib.pyplot as plt
from segy.schema import HeaderField
from segy.standards import get_segy_standard

from mdio import segy_to_mdio
from mdio.builder.schemas.v1.units import TimeUnitModel
from mdio.builder.template_registry import get_template

shot_trace_headers = [
    HeaderField(name="shot_point", byte=9, format="int32"),
    HeaderField(name="channel", byte=13, format="int32"),
]

rev1_segy_spec = get_segy_standard(1.0)
soda_lake_segy_spec = rev1_segy_spec.customize(trace_header_fields=shot_trace_headers)

mdio_template = get_template("StreamerShotGathers2D")
unit_ms = TimeUnitModel(time="ms")
mdio_template.add_units({"time": unit_ms})

segy_to_mdio(
    input_path=input_url,
    output_path="soda_lake.mdio",
    segy_spec=soda_lake_segy_spec,
    mdio_template=mdio_template,
    overwrite=True,
)

It only took a few seconds to ingest, since this is a very small file.

However, MDIO scales up to TB (that’s ~1,000 GB) sized volumes!

Querying Metadata¶

Now let’s look at the file text header saved in the segy_file_header metadata variable.

print(dataset["segy_file_header"].attrs["textHeader"])

C 1 CLIENT                        COMPANY                       CREW NO         
C 2 LINE            AREA                        MAP ID                          
C 3 REEL NO           DAY-START OF REEL     YEAR      OBSERVER                  
C 4 INSTRUMENT: MFG            MODEL            SERIAL NO                       
C 5 DATA TRACES/RECORD        AUXILIARY TRACES/RECORD         CDP FOLD          
C 6 SAMPLE INTERVAL         SAMPLES/TRACE       BITS/IN      BYTES/SAMPLE       
C 7 RECORDING FORMAT        FORMAT THIS REEL        MEASUREMENT SYSTEM          
C 8 SAMPLE CODE: FLOATING PT     FIXED PT     FIXED PT-GAIN     CORRELATED      
C 9 GAIN  TYPE: FIXED     BINARY     FLOATING POINT     OTHER                   
C10 FILTERS: ALIAS     HZ  NOTCH     HZ  BAND     -     HZ  SLOPE    -    DB/OCT
C11 SOURCE: TYPE            NUMBER/POINT        POINT INTERVAL                  
C12     PATTERN:                           LENGTH        WIDTH                  
C13 SWEEP: START     HZ  END     HZ  LENGTH      MS  CHANNEL NO     TYPE        
C14 TAPER: START LENGTH       MS  END LENGTH       MS  TYPE                     
C15 SPREAD: OFFSET        MAX DISTANCE        GROUP INTERVAL                    
C16 GEOPHONES: PER GROUP     SPACING     FREQUENCY     MFG          MODEL       
C17     PATTERN:                           LENGTH        WIDTH                  
C18 TRACES SORTED BY: RECORD     CDP     OTHER                                  
C19 AMPLITUDE RECOVERY: NONE      SPHERICAL DIV      AGC     OTHER              
C20 MAP PROJECTION                      ZONE ID       COORDINATE UNITS          
C21 PROCESSING:                                                                 
C22 PROCESSING:                                                                 
C23                                                                             
C24                                                                             
C25                                                                             
C26                                                                             
C27                                                                             
C28                                                                             
C29                                                                             
C30                                                                             
C31                                                                             
C32                                                                             
C33                                                                             
C34                                                                             
C35                                                                             
C36                                                                             
C37                                                                             
C38                                                                             
C39                                                                             
C40 END EBCDIC                                                                  

Since we saved the binary header, we can look at that as well.

dataset["segy_file_header"].attrs["binaryHeader"]

{'job_id': 18909,
 'line_num': 1,
 'reel_num': 1,
 'data_traces_per_ensemble': 957,
 'aux_traces_per_ensemble': 1,
 'sample_interval': 2000,
 'orig_sample_interval': 2000,
 'samples_per_trace': 2000,
 'orig_samples_per_trace': 2000,
 'data_sample_format': 1,
 'ensemble_fold': 0,
 'trace_sorting_code': 1,
 'vertical_sum_code': 1,
 'sweep_freq_start': 0,
 'sweep_freq_end': 0,
 'sweep_length': 0,
 'sweep_type_code': 4,
 'sweep_trace_num': 0,
 'sweep_taper_start': 0,
 'sweep_taper_end': 0,
 'taper_type_code': 3,
 'correlated_data_code': 1,
 'binary_gain_code': 2,
 'amp_recovery_code': 1,
 'measurement_system_code': 2,
 'impulse_polarity_code': 0,
 'vibratory_polarity_code': 0,
 'fixed_length_trace_flag': 1,
 'num_extended_text_headers': 0,
 'segy_revision_major': 0,
 'segy_revision_minor': 0}

Fetching Data and Plotting¶

We keep one component (transduction_const_mantissa == 2), clip by the 98th percentile, and plot against source-receiver offset. Channel order follows the 2D receiver patch, not offset.

Since MDIO v1 we are using Xarray under the hood, so we can use its convenient indexing. It also handles the plotting with proper dimension coordinate labels.

import numpy as np

COMPONENT_A = 2

shot = dataset.sel(shot_point=7733)
mantissa = shot.headers.to_numpy()["transduction_const_mantissa"]
comp = shot.isel(channel=mantissa == COMPONENT_A)

offset = np.hypot(
    comp.group_coord_x.to_numpy() - float(comp.source_coord_x),
    comp.group_coord_y.to_numpy() - float(comp.source_coord_y),
)
comp = comp.assign_coords(offset=("channel", offset)).sortby("offset")

amp = comp["amplitude"]
clip = float(np.percentile(np.abs(amp.to_numpy()), 98))

amp.plot(x="offset", y="time", vmin=-clip, vmax=clip, cmap="gray_r", yincrease=False);

../_images/4b9ca4ae15a67eb520e133a1029a910a34177e96f97678e0d0c5a98df17a6c87.png

Receiver coordinates in map view, then the same points colored by amplitude at 1,000 ms.

fig, ax = plt.subplots(figsize=(6, 5))
ax.scatter(comp.group_coord_x, comp.group_coord_y, s=16, c="0.3", label="receivers")
ax.scatter(
    float(comp.source_coord_x),
    float(comp.source_coord_y),
    marker="*",
    s=180,
    c="tab:red",
    label="source",
    zorder=3,
)
ax.set_xlabel("group_coord_x")
ax.set_ylabel("group_coord_y")
ax.set_aspect("equal", adjustable="box")
ax.legend()
ax.set_title("component A receiver patch");

../_images/7d70530e547cfe03ec6d2b2b8c0968cf251b37fee36a03fcfed4479c888ed8d3.png

Color the same map by amplitude at 1,000 ms.

twt = amp.sel(time=1000)

fig, ax = plt.subplots(figsize=(6, 5))
sc = ax.scatter(
    comp.group_coord_x,
    comp.group_coord_y,
    c=twt,
    s=16,
    cmap="gray_r",
    vmin=-clip,
    vmax=clip,
)
ax.scatter(
    float(comp.source_coord_x),
    float(comp.source_coord_y),
    marker="*",
    s=180,
    c="tab:red",
    label="source",
    zorder=3,
)
ax.set_xlabel("group_coord_x")
ax.set_ylabel("group_coord_y")
ax.set_aspect("equal", adjustable="box")
ax.legend()
plt.colorbar(sc, ax=ax, label="amplitude @ 1000 ms");

../_images/7233345b3c25194323a4f56995615ee084727d517699bf78891280ce650c5eb9.png

MDIO to SEG-Y Conversion¶

Finally, let’s demonstrate going back to SEG-Y.

We will use the convenient mdio_to_segy function and write it out as a round-trip file.

The output spec can be modified if we want to write things to different byte locations, etc, but we will use the same one as before.

from mdio import mdio_to_segy

mdio_to_segy(
    input_path="soda_lake.mdio",
    output_path="soda_lake_roundtrip.sgy",
    segy_spec=soda_lake_segy_spec,
)

Validate Round-Trip SEG-Y File¶

We can validate if the round-trip SEG-Y file is matching the original using TGSAI/segy.

Step by step:

Open original file
Open round-trip file
Compare text headers
Compare binary headers
Compare 100 random headers and traces

import numpy as np
from segy import SegyFile

original_segy = SegyFile(input_url)
roundtrip_segy = SegyFile("soda_lake_roundtrip.sgy")

# Compare text header
assert original_segy.text_header == roundtrip_segy.text_header

# Compare bin header
assert original_segy.binary_header == roundtrip_segy.binary_header

# Compare 100 random trace headers and samples.
# IBM float32 cannot preserve denormals through IEEE decode/encode.
rng = np.random.default_rng()
rand_indices = rng.integers(low=0, high=original_segy.num_traces, size=100)
for idx in rand_indices:
    np.testing.assert_equal(original_segy.header[idx], roundtrip_segy.header[idx])
    np.testing.assert_allclose(original_segy.sample[idx], roundtrip_segy.sample[idx], rtol=0, atol=1e-37)

print("Files identical!")

Files identical!

Get Started in 10 Minutes¶

Defining the SEG-Y Dataset¶

Ingesting to MDIO¶

Opening the Ingested MDIO File¶

Querying Metadata¶

Fetching Data and Plotting¶

Query Headers¶

MDIO to SEG-Y Conversion¶

Validate Round-Trip SEG-Y File¶

Get Started in 10 Minutes¶

Defining the SEG-Y Dataset¶

Ingesting to MDIO¶

Opening the Ingested MDIO File¶

Querying Metadata¶

MDIO Grid, Dimensions, and Related Attributes¶

Fetching Data and Plotting¶

Query Headers¶

MDIO to SEG-Y Conversion¶

Validate Round-Trip SEG-Y File¶