Hi!
As Frankie says, the intent of the optional src_eff parameter on regular Satellites in AutomateDV is to support business-effectivity tracking for those records. This only applies to regular Sats, not Effectivity Sats.
If we take a step back and assume we aren’t using AutomateDV, it helps to look at this in a tool-agnostic way.
Late-arriving data forces us to reassess “what we knew, and when we knew it”. We can think of this as two timelines (or alternate universes, for fun):
Timeline 1: What we believed to be true before the late-arriving data
Timeline 2: What we later discovered to have been true once that data arrived
A few important things to note/ take away from this:
- The business may have taken action based on Timeline 1, even though Timeline 2 tells us that information was incomplete or incorrect.
- Changes to business-effective timelines can have downstream knock-on effects.
- Rewriting history is generally ill-advised in the Raw Vault.
Starting with point 3: the Raw Vault must remain exactly that: raw. This is why regular Satellites in AutomateDV optionally support src_eff: it allows you to capture an additional business-effective timestamp alongside load time, so you can reason about multiple timelines later. It is not intended to overwrite or reorder history in place.
Late-arriving or out-of-sequence data should therefore be loaded as-is, preserving both load timestamps and business-effective timestamps. Effectivity Satellites allow you to record “when a relationship was discovered” versus “when it was actually valid”, without rewriting history.
Any reconciliation between those timelines (for example deciding which version of reality to report on) belongs in the Business Vault or downstream presentation layers, not in the Raw Vault itself.
With Effectivity Satellites in particular, points 1 and 2 become more visible. For example, consider an Effectivity Satellite tracking an employee’s department assignment. HR may only discover after the fact that an employee moved departments, potentially after payroll has already closed.
That late discovery can impact cost allocation, reporting, and other downstream processes, even though the Raw Vault correctly reflects both what was known at the time and what was later discovered.
All of this is to say: track the right metadata in the Raw Vault so downstream business rules can be “late-arriving aware”. You don’t “fix” late-arriving data in the Raw Vault because there is nothing to fix. The reality is that there were always two timelines.
How those timelines are reconciled or presented to the business is ultimately a business decision, but the first step is capturing them separately and faithfully.
Once you’ve done that, you can have an informed conversation with the business about how they want to handle these scenarios, armed with both the knowledge and the data from your newly discovered multiverse.
I hope that helps you sleep a little more easily at night 