Get Started in 10 Minutes

Altay Sansal

Sep 09, 2026

9 min read

In this page we will be showing basic capabilities of MDIO.

For demonstration purposes, we will ingest the remote Teapot Dome open-source dataset. The dataset details and licensing can be found at the SEG Wiki.

We are using the 3D seismic stack dataset named filt_mig.sgy.

The full HTTP link for the dataset (hosted on AWS): http://s3.amazonaws.com/teapot/filt_mig.sgy

Warning

For plotting and remote ingestion the notebook requires Matplotlib and aiohttp as a dependency. Please install it before executing using pip install matplotlib aiohttp or conda install matplotlib aiohttp.

Defining the SEG-Y Dataset

Since MDIO 0.8 we can directly ingest remote SEG-Y files! The file is 386 MB in size. To make the header scan performant we can also set up an environment variable for MDIO. See here for when to use this: Buffered Reads in Ingestion.

The dataset is irregularly shaped, however it is padded to a rectangle with zeros (dead traces). We will see that later at the live mask plotting.

The following environment variables are essential here:

  • MDIO__IMPORT__CLOUD_NATIVE tells MDIO to do buffered reads for headers due to remote file.

  • MDIO__IMPORT__SAVE_SEGY_FILE_HEADER makes MDIO save the SEG-Y specific file headers (text, binary) which is not strictly necessary for consumption and is disabled by default.

import os

os.environ["MDIO__IMPORT__CLOUD_NATIVE"] = "true"
os.environ["MDIO__IMPORT__SAVE_SEGY_FILE_HEADER"] = "true"

input_url = "http://s3.amazonaws.com/teapot/filt_mig.sgy"

Ingesting to MDIO

To do this, we can use the convenient SEG-Y to MDIO converter.

The inline and crossline values are located at bytes 181 and 185. Note that this doesn’t match any SEG-Y standards.

MDIO uses TGSAI/segy to parse the SEG-Y; the field names conform to its canonical keys defined in SEGY Binary Header Keys and SEGY Trace Header Keys. Since MDIO v1 we also introduced templates for common seismic data types. For instance, we will be using the PostStack3DTime template here, which expects the same canonical keys.

We will also specify the units for the time domain. The spatial units will be automatically parsed from SEG-Y binary header. However, there may be a case where it is corrupt in the file, for that see the Fixing X/Y Units Issues section.

In summary, we will use the byte locations as defined for ingestion.

import matplotlib.pyplot as plt
from segy.schema import HeaderField
from segy.standards import get_segy_standard

from mdio import segy_to_mdio
from mdio.builder.schemas.v1.units import TimeUnitModel
from mdio.builder.template_registry import get_template

teapot_trace_headers = [
    HeaderField(name="inline", byte=181, format="int32"),
    HeaderField(name="crossline", byte=185, format="int32"),
    HeaderField(name="cdp_x", byte=189, format="int32"),
    HeaderField(name="cdp_y", byte=193, format="int32"),
]

rev0_segy_spec = get_segy_standard(0)
teapot_segy_spec = rev0_segy_spec.customize(trace_header_fields=teapot_trace_headers)

mdio_template = get_template("PostStack3DTime")
unit_ms = TimeUnitModel(time="ms")
mdio_template.add_units({"time": unit_ms})

segy_to_mdio(
    input_path=input_url,
    output_path="filt_mig.mdio",
    segy_spec=teapot_segy_spec,
    mdio_template=mdio_template,
    overwrite=True,
)

---------------------------------------------------------------------------
ClientResponseError                       Traceback (most recent call last)
File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/fsspec/implementations/http.py:444, in HTTPFileSystem._info(self, url, **kwargs)
    442 try:
    443     info.update(
--> 444         await _file_info(
    445             self.encode_url(url),
    446             size_policy=policy,
    447             session=session,
    448             **self.kwargs,
    449             **kwargs,
    450         )
    451     )
    452     if info.get("size") is not None:

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/fsspec/implementations/http.py:860, in _file_info(url, session, size_policy, **kwargs)
    859 async with r:
--> 860     r.raise_for_status()
    862     if "Content-Length" in r.headers:
    863         # Some servers may choose to ignore Accept-Encoding and return
    864         # compressed content, in which case the returned size is unreliable.

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/aiohttp/client_reqrep.py:655, in ClientResponse.raise_for_status(self)
    653     self.release()
--> 655 raise ClientResponseError(
    656     self.request_info,
    657     self.history,
    658     status=self.status,
    659     message=self.reason,
    660     headers=self.headers,
    661 )

ClientResponseError: 403, message='Forbidden', url='http://s3.amazonaws.com/teapot/filt_mig.sgy'

The above exception was the direct cause of the following exception:

FileNotFoundError                         Traceback (most recent call last)
Cell In[2], line 23
     19 mdio_template = get_template("PostStack3DTime")
     20 unit_ms = TimeUnitModel(time="ms")
     21 mdio_template.add_units({"time": unit_ms})
     22 
---> 23 segy_to_mdio(
     24     input_path=input_url,
     25     output_path="filt_mig.mdio",
     26     segy_spec=teapot_segy_spec,

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/mdio/converters/segy.py:72, in segy_to_mdio(segy_spec, mdio_template, input_path, output_path, overwrite, grid_overrides, segy_header_overrides)
     68 typed_grid_overrides = _coerce_grid_overrides(grid_overrides)
     70 from mdio.ingestion.segy.pipeline import segy_to_mdio as _ingest_segy_to_mdio  # noqa: PLC0415
---> 72 return _ingest_segy_to_mdio(
     73     segy_spec=segy_spec,
     74     mdio_template=mdio_template,
     75     input_path=input_path,
     76     output_path=output_path,
     77     overwrite=overwrite,
     78     grid_overrides=typed_grid_overrides,
     79     segy_header_overrides=segy_header_overrides,
     80 )

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/mdio/ingestion/segy/pipeline.py:154, in segy_to_mdio(segy_spec, mdio_template, input_path, output_path, overwrite, grid_overrides, segy_header_overrides)
    146 output_path = _resolve_output_path(output_path, overwrite)
    148 segy_file_kwargs: SegyFileArguments = {
    149     "url": input_path.as_posix(),
    150     "spec": segy_spec,
    151     "settings": SegyFileSettings(storage_options=input_path.storage_options),
    152     "header_overrides": segy_header_overrides,
    153 }
--> 154 segy_file_info = get_segy_file_info(segy_file_kwargs)
    156 spatial_unit = get_spatial_coordinate_unit(segy_file_info)
    157 units = resolve_units(mdio_template, spatial_unit)

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/mdio/segy/file.py:161, in get_segy_file_info(segy_file_kwargs)
    152 def get_segy_file_info(segy_file_kwargs: SegyFileArguments) -> SegyFileInfo:
    153     """Reads information from a SEG-Y file.
    154 
    155     Args:
   (...)    159         SegyFileInfo containing number of traces, sample labels, and header info.
    160     """
--> 161     segy_file = SegyFileWrapper(**segy_file_kwargs)
    162     num_traces = segy_file.num_traces
    163     sample_labels = segy_file.sample_labels

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/mdio/segy/file.py:149, in SegyFileWrapper.__init__(self, url, spec, settings, header_overrides)
    142 args = SegyFileArguments(
    143     url=url,
    144     spec=spec,
    145     settings=settings,
    146     header_overrides=header_overrides,
    147 )
    148 _start_asyncio_loop(args)
--> 149 super().__init__(**args)

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/segy/file.py:75, in SegyFile.__init__(self, url, spec, settings, header_overrides)
     72 self.header_overrides = header_overrides or SegyHeaderOverrides()
     74 self.fs, self.url = url_to_fs(url, **self.settings.storage_options)
---> 75 self._info = self.fs.info(self.url)
     77 self.spec = self._initialize_spec(spec)
     78 self._update_spec()

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/fsspec/asyn.py:118, in sync_wrapper.<locals>.wrapper(*args, **kwargs)
    115 @functools.wraps(func)
    116 def wrapper(*args, **kwargs):
    117     self = obj or args[0]
--> 118     return sync(self.loop, func, *args, **kwargs)

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/fsspec/asyn.py:103, in sync(loop, func, timeout, *args, **kwargs)
    101     raise FSTimeoutError from return_result
    102 elif isinstance(return_result, BaseException):
--> 103     raise return_result
    104 else:
    105     return return_result

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/fsspec/asyn.py:56, in _runner(event, coro, result, timeout)
     54     coro = asyncio.wait_for(coro, timeout=timeout)
     55 try:
---> 56     result[0] = await coro
     57 except Exception as ex:
     58     result[0] = ex

File ~/checkouts/readthedocs.org/user_builds/mdio-python/envs/stable/lib/python3.13/site-packages/fsspec/implementations/http.py:457, in HTTPFileSystem._info(self, url, **kwargs)
    454     except Exception as exc:
    455         if policy == "get":
    456             # If get failed, then raise a FileNotFoundError
--> 457             raise FileNotFoundError(url) from exc
    458         logger.debug("", exc_info=exc)
    460 return {"name": url, "size": None, **info, "type": "file"}

FileNotFoundError: http://s3.amazonaws.com/teapot/filt_mig.sgy

It only took a few seconds to ingest, since this is a very small file.

However, MDIO scales up to TB (that’s ~1,000 GB) sized volumes!

Opening the Ingested MDIO File

Let’s open the MDIO file with the open_mdio function. This will return a pretty xarray representation with our standardized format.

from mdio import open_mdio

dataset = open_mdio("filt_mig.mdio")
dataset

Querying Metadata

Now let’s look at the file text header saved in the segy_file_header metadata variable.

print(dataset["segy_file_header"].attrs["textHeader"])

Since we saved the binary header, we can look at that as well.

dataset["segy_file_header"].attrs["binaryHeader"]

Fetching Data and Plotting

Now we will demonstrate getting an inline from MDIO.

Since MDIO v1 we are using Xarray under the hood, so we can use its convenient indexing. It also handles the plotting with proper dimension coordinate labels.

MDIO stores summary statistics. We can calculate the standard deviation (std) value of the dataset to adjust the gain.

from mdio.builder.schemas.v1.stats import SummaryStatistics

stats = SummaryStatistics.model_validate_json(dataset["amplitude"].attrs["statsV1"])
std = ((stats.sum_squares / stats.count) - (stats.sum / stats.count) ** 2) ** 0.5

il_dataset = dataset.sel(inline=278)
il_amp = il_dataset["amplitude"].T
il_amp.plot(vmin=-2 * std, vmax=2 * std, cmap="gray_r", yincrease=False);

Let’s do the same with a time slice.

We will display two-way-time at 1,000 ms.

Note that since we parse the X/Y coordinates, we can plot time slice in real world coordinates.

twt_data = dataset["amplitude"].sel(time=1000)
twt_data.plot(vmin=-2 * std, vmax=2 * std, cmap="gray_r", x="cdp_x", y="cdp_y");

We can also overlay live mask with the time slice. However, in this example, the dataset is zero-padded.

The live trace mask will always show True (yellow).

trace_mask = dataset.trace_mask[:]

twt_data.plot(vmin=-2 * std, vmax=2 * std, cmap="gray_r", x="cdp_x", y="cdp_y", alpha=0.5, figsize=(8, 5))
trace_mask.plot(vmin=0, vmax=1, x="cdp_x", y="cdp_y", alpha=0.5);

Query Headers

We can query headers for the whole dataset very quickly because they are separated from the seismic wavefield.

Let’s get all the headers for X and Y coordinates in this dataset.

Note that the header maps will still share the geometry/grid of the dataset!

The compute property fetches the lazily opened MDIO values.

dataset.headers["cdp_x"].compute()

dataset.headers["cdp_y"].compute()

As we mentioned before, we can also get specific dataset slices of headers while fetching a slice.

Let’s fetch a crossline; we are still using some previous parameters.

Since the sliced dataset contains the headers as well, we can plot the headers on top of the image.

Full headers can be mapped and plotted as well, but we won’t demonstrate that here.

xl_dataset = dataset.sel(crossline=100)  # slices everything available in MDIO dataset!

cdp_x_header = xl_dataset["cdp_x"]
cdp_y_header = xl_dataset["cdp_y"]
image = xl_dataset["amplitude"].T

# Build plot from here
gs_kw = {"height_ratios": (1, 8)}
fig, (hdr_ax, img_ax) = plt.subplots(2, 1, figsize=(6, 6), gridspec_kw=gs_kw, sharex="all")
hdr_ax2 = hdr_ax.twinx()

cdp_x_header.plot(ax=hdr_ax, c="red")
cdp_y_header.plot(ax=hdr_ax2, c="black")
image.plot(ax=img_ax, vmin=-2 * std, vmax=2 * std, cmap="gray_r", yincrease=False, add_colorbar=False)
img_ax.set_title("")
hdr_ax.set_xlabel("")

plt.tight_layout()

MDIO to SEG-Y Conversion

Finally, let’s demonstrate going back to SEG-Y.

We will use the convenient mdio_to_segy function and write it out as a round-trip file.

The output spec can be modified if we want to write things to different byte locations, etc, but we will use the same one as before.

from mdio import mdio_to_segy

mdio_to_segy(
    input_path="filt_mig.mdio",
    output_path="filt_mig_roundtrip.sgy",
    segy_spec=teapot_segy_spec,
)

Validate Round-Trip SEG-Y File

We can validate if the round-trip SEG-Y file is matching the original using TGSAI/segy.

Step by step:

  • Open original file

  • Open round-trip file

  • Compare text headers

  • Compare binary headers

  • Compare 100 random headers and traces

import numpy as np
from segy import SegyFile

original_segy = SegyFile(input_url)
roundtrip_segy = SegyFile("filt_mig_roundtrip.sgy")

# Compare text header
assert original_segy.text_header == roundtrip_segy.text_header

# Compare bin header
assert original_segy.binary_header == roundtrip_segy.binary_header

# Compare 100 random trace headers and traces
rng = np.random.default_rng()
rand_indices = rng.integers(low=0, high=original_segy.num_traces, size=100)
for idx in rand_indices:
    np.testing.assert_equal(original_segy.trace[idx], roundtrip_segy.trace[idx])

print("Files identical!")