iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
A Delta table stores its row data in Parquet files, but it is not simply every Parquet file in a folder. In the usual filesystem-based layout, a transaction log records commits and defines which data files and metadata make up each table version. Delta-aware software uses that log to read the table’s authoritative snapshot.
What files make up a Delta table?
Think of a Delta table as two cooperating layers:
- Data layer: Parquet files store rows in a columnar format.
- Transaction and metadata layer: the Delta transaction log records table changes and describes the state of each version, including which data files belong to it.
In the usual filesystem-based layout, the table directory contains Parquet data files alongside a _delta_log/ directory. The exact files can change as the table is updated; the log, rather than a directory listing alone, determines the files that represent the current table. Delta Lake’s FAQ describes this as versioned Parquet files plus a separate transaction log.
Why does Delta need a transaction log?
A plain Parquet reader can open individual Parquet files, but it does not by itself know which files make up the current Delta snapshot. Updates and deletes can produce a new table state that refers to the appropriate data files; old or superseded files may still exist in the directory. A Delta-aware reader consults the log to identify the table’s committed state instead of treating every file it finds as live data.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallThe log records commits and versions, enabling Delta Lake’s transaction behavior and version-aware access. The project overview describes capabilities such as ACID transactions, schema enforcement, batch and streaming use, and update, delete, and merge operations. These capabilities depend on a client that understands the table and its enabled protocol features; the Parquet files alone do not provide them. See the Delta Lake overview.
#1 Best Overall
- Easily store and access 2TB to content on the go with the Seagate Portable Drive, a USB external hard drive
- Designed to work with Windows or Mac computers, this external hard drive makes backup a snap just drag and drop
- To get set up, connect the portable hard drive to a computer for automatic recognition no software required
- This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable
- The available storage capacity may vary.
How does time travel work, and what can limit it?
A Delta-aware client can use the transaction log to query an earlier committed version. The Delta Lake quick start demonstrates time travel and documents a streaming-write scenario in which the log supports exactly-once processing, including alongside concurrent batch queries or streams. Those behaviors apply to supported Delta operations, not to opening the Parquet files as an ordinary directory.
Older versions remain queryable only while the needed log information and data files are available. Delta Lake’s documentation lists a default of 30 days for transaction-log retention and table history; these are configurable defaults, not a guarantee that any particular table retains exactly 30 days. The table properties documentation describes delta.logRetentionDuration, and the utility documentation explains history and VACUUM. VACUUM removes data files outside the applicable retention period; if an older version needs a file that has been removed, time travel to that version is no longer possible. Check the table’s settings and cleanup history before assuming a version is still available.
Rank #2
- Easily store and access 5TB of content on the go with the Seagate portable drive, a USB external hard Drive
- Designed to work with Windows or Mac computers, this external hard drive makes backup a snap just drag and drop
- To get set up, connect the portable hard drive to a computer for automatic recognition software required
- This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable
- The available storage capacity may vary.
Can any Parquet reader read a Delta table correctly?
No. A generic Parquet reader may return data from files it can open, but it does not necessarily apply the Delta log’s snapshot selection, metadata, or protocol-directed transformations. Delta Kernel documentation describes readers obtaining file status from a scan and applying protocol- and metadata-directed transformations to physical Parquet data. That is why “Parquet plus a log” is a useful mental model, but not a claim that every Parquet tool can reproduce the logical Delta table. See Delta Kernel documentation.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Delta tables also declare supported protocol features. An application must understand the features enabled for a particular table to read or write it correctly. Before accessing a table from another engine, verify that engine’s Delta protocol and feature support, rather than checking only whether it supports Parquet. The protocol versioning documentation explains the compatibility requirement.
Rank #3
- Easily store and access 1TB to content on the go with the Seagate Portable Drive, a USB external hard drive.Specific uses: Personal
- Designed to work with Windows or Mac computers, this external hard drive makes backup a snap just drag and drop. Reformatting may be required for Mac
- To get set up, connect the portable hard drive to a computer for automatic recognition no software required
- This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable
- The available storage capacity may vary.
How do you create a Delta table from Parquet?
Write data in Delta format
The documented Spark quick start creates a table by writing a DataFrame with the delta format. In the documented Spark SQL creation flow, delta can be used in place of parquet. See the quick start for the example and supported setup.
Convert an existing Parquet directory
Delta Lake’s conversion utility can create a transaction log for an existing Parquet directory. The documented process lists the existing files, creates a log that tracks them, and infers the schema from Parquet footers. For partitioned data, an explicit partition-column schema may be required. In other words, the Parquet data can predate the Delta table: the log is created to define and maintain the table state. Details are in the utility commands documentation.
Rank #4
- Easily store and access 4TB of content on the go with the Seagate Portable Drive, a USB external hard drive.Specific uses: Personal
- Designed to work with Windows or Mac computers, this external hard drive makes backup a snap just drag and drop
- To get set up, connect the portable hard drive to a computer for automatic recognition no software required
- This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable
- The available storage capacity may vary.
What changes with catalog-managed tables?
The filesystem layout and commit description above are the usual model, not a universal rule. Delta Lake documentation for version 4.0.1 and later describes a catalogManaged feature: a managing catalog coordinates commits, direct filesystem-based access is unsupported, and the protocol version is upgraded. For a catalog-managed table, follow the managing catalog’s supported access path rather than assuming that inspecting its directory will expose a usable Delta table. See catalog-managed tables.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Quick Recap
Best Value
- Plug-and-play expandability
- SuperSpeed USB 3.2 Gen 1 (5Gbps)
Delta table vs. plain Parquet directory
| Question | Delta table | Plain Parquet directory |
|---|---|---|
| What defines the current table data? | The transaction log identifies the committed snapshot and its files. | There is no Delta transaction log defining a versioned snapshot. |
| What transaction and version behavior is available? | Delta-aware clients use logged commits for transaction behavior and version-aware access, subject to supported features and available history. | Parquet files alone do not provide Delta commits or Delta time travel. |
| What must a reader support? | Parquet plus the relevant Delta protocol features. | Parquet support to read the files; no Delta protocol is present. |
| What determines whether older data remains accessible? | Retention settings and cleanup, including whether VACUUM removed files required by an older version. | No Delta history is defined; file retention is managed outside Delta. |
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
