Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstalliTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To replicate MySQL into BigQuery with Sling, configure a named connection for each system, define the tables and replication mode in a YAML file, then run it with the Sling CLI. Use full-refresh for an initial copy, timestamp-and-key incremental replication for suitable recurring updates, or transaction-log CDC when inserts, updates, and deletes must be captured. These modes have different requirements and should not be treated as interchangeable.
What you need before setting up Sling
- A MySQL database that Sling can reach and a Google Cloud project with a BigQuery dataset for the destination.
- Sling installed on the machine or environment that will run the replication. The official Sling guide documents installation options for macOS, Linux, Windows, and Python; follow its current instructions for your platform.
- A Google service-account key file for the BigQuery connection. Sling’s guide says the account needs the BigQuery Data Editor and BigQuery Job User roles on the target project.
- Credentials stored securely. Avoid placing passwords or service-account secrets in YAML that is shared or checked into source control.
After installation, confirm the CLI is available with sling --version. Configure named MySQL and BigQuery connections using Sling’s current connection instructions, then test both connections before attempting a replication.
Choose the replication mode
The right mode depends on what must happen to existing rows, changed rows, and deleted rows. Sling’s replication guide describes full-refresh and key-based incremental runs; its CDC documentation describes transaction-log capture.
| Mode | How it works | Use it when | Important limitation or requirement |
|---|---|---|---|
full-refresh |
Loads the selected source data as a fresh full copy. | You are performing an initial load or refreshing a small table. | Check the target and overwrite behavior for the specific mode and Sling version before using it on production data. |
incremental |
Polls for qualifying records using an update key, with a primary key used for the intended merge behavior. | You need recurring batch updates and have reliable keys. | It is not transaction-log CDC. Do not assume it captures deletes; the described approach depends on a timestamp or monotonic key that reliably identifies new or changed records. |
| CDC | Reads MySQL’s binary log to capture inserts, updates, and deletes, with an initial load followed by ongoing change capture. | Changes, including deletes, need to be propagated from MySQL. | Verify MySQL binlog prerequisites and current Sling plan requirements. Sling documents CDC as requiring a CLI Pro Max token or an Advanced Platform Plan. |
In Sling’s terminology, CDC reads the source database’s transaction log for row-level changes. That is distinct from incremental polling based on an update key: a timestamp-based run can miss changes if the key is not maintained reliably, and it does not provide the delete capture that Sling documents for CDC.
#1 Best Overall
Configure and run a replication
1. Define the source, target, and streams
Create a YAML replication file containing source, target, optional defaults, and streams. Use the names of the connections you configured. A schema wildcard such as demo.* selects all tables in that schema; the target template {stream_table} can retain each table’s name.
For example, the structure can look like this; replace the connection names, schema, and table settings with values appropriate to your environment:
Rank #2
source: mysql_source
target: bigquery_target
defaults:
mode: full-refresh
object: "{stream_table}"
streams:
demo.*: {}
YAML indentation and supported fields matter. Treat this as a structural illustration rather than a complete credential file: connection secrets belong in Sling’s connection configuration, not in a shared replication file. Sling’s replication reference explains shared defaults and stream configuration; explicit stream settings can override defaults.
Recommended Free Tools
2. Load the initial data
For an initial copy, set the applicable stream mode to full-refresh. Before running it against an existing BigQuery dataset, confirm which destination tables will be created or replaced under the mode and Sling version you are using. Do not assume that a mode’s effect on production tables is safe without checking those details.
3. Set up recurring incremental updates, if appropriate
For key-based incremental runs, configure an update_key that reliably reflects new or changed source records and a primary_key that uniquely identifies rows for the intended merge behavior. Sling’s example uses a timestamp as the update key and an order ID as the primary key. Those are examples, not universal choices: select columns based on the guarantees in your own schema and application.
If the source has no reliable timestamp or monotonic update key, the described incremental approach is limited to full-refresh or snapshot modes. If deletes must reach BigQuery, select and validate CDC rather than treating timestamp polling as equivalent.
4. Run the YAML file
Run the replication from a shell with:
sling run -r path/to/replication.yamlReplace the path with the location of your file. Check the run output and BigQuery destination to confirm that the intended streams completed and that rows landed in the expected dataset and tables.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →How Sling loads data into BigQuery
Sling’s guide describes bulk loading through BigQuery load jobs, rather than inserting every row individually. When a Google Cloud Storage bucket is configured, Sling can stage batches there. Without a bucket, the guide says Sling stages data locally and still loads it in bulk. The staging path affects setup and where intermediate data is held; it does not change the distinction between full-refresh, incremental, and CDC semantics.
Best Value
Sling reports two example measurements from its 2026 guide: 80,000 rows in about 23 seconds and 2,500 rows inserted in 44 seconds. These are vendor-reported results from a described run against MySQL 8.4 and a live BigQuery dataset, not an independent benchmark or a prediction for another workload. Actual elapsed time depends on the data, configuration, environment, and service conditions.
Schedule and operate recurring runs
Sling’s guide shows cron as a basic way to schedule a working replication. A scheduler alone does not provide production monitoring or recovery. Decide how the deployment will handle logs, failures, retries, alerting, and reruns, and make sure operators can tell whether each expected run completed.
Before enabling an ongoing schedule, check these items:
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Quick Recap
- Confirm the service account accesses the intended Google Cloud project and dataset and has the documented BigQuery roles.
- Review the MySQL tables selected, including the scope of any schema wildcard, and confirm the target dataset and table-name template.
- Verify whether shared defaults or stream-specific settings determine the mode and destination for each table.
- Choose the mode according to the source’s change semantics. In particular, confirm whether deletes must propagate and whether the chosen approach supports that requirement.
- For CDC, verify current plan eligibility and MySQL binary-log prerequisites with Sling’s current documentation before deployment.
- Plan logging, monitoring, alerting, retries, and recovery for the environment; a sample cron schedule does not establish those controls.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
