Book a consultation

Case studies

Three problems that hold growing companies back, and how we solve them: files that cannot be traced to the records they belong to, data that cannot be trusted, and storage that costs more than it should.

01Case study

Generating Unique Identifiers for Unstructured Data

Utilizing AI to identify preexisting unique identifiers and primary keys across established relational databases.

  • Software architecture
  • Data migration
  • AI
Files, each with a unique IDDocument and vector indexRelational databasesKeys found by AI
Discuss a similar project

The problem

Contracts, invoices, scanned forms, images and emails tend to live in shared drives and inboxes, with nothing that ties them to the customer, order or asset they describe.

Older relational databases often keep their real keys only by convention: columns that behave like primary and foreign keys but were never declared. Without reliable identifiers on both sides, files cannot be joined to records, duplicates multiply and every migration starts with guesswork.

Our approach

  1. Store every file as an object with its own ID

    Raw files move to object storage in a flat structure instead of nested folders. Each object receives a system-generated unique identifier (UUID) that stays with it wherever it is used.

  2. Attach metadata to each object

    The identifier carries custom metadata with it: file size, timestamp, source system, tags and business categories, so every file can be found, governed and audited.

  3. Index content by key and by meaning

    A document database records every object under its _id primary key. Where people need to search by meaning rather than by key, a vector index is linked to the same identifier.

  4. Extract content with AI and keep the link

    Language, vision or LLM-based extraction pipelines read each file and pull out names, dates, amounts and references, always tied back to the file’s identifier.

  5. Find the keys your databases already use

    Profiling measures how unique, complete and overlapping every column is. AI-assisted analysis of names, formats and values ranks the likely primary keys and the undeclared relationships between tables, and our engineers validate each one before anything depends on it.

  6. Link files to records

    References extracted from files are matched to the validated keys, so each document points to the record it belongs to. Matches below an agreed confidence level go to a review queue instead of being linked automatically.

What you receive

  • Identifier and metadata standard for unstructured content
  • Object storage and indexing architecture
  • Extraction pipeline design with confidence thresholds and a review queue
  • Key discovery report: candidate primary and foreign keys, the evidence for each and whether it was confirmed
  • Validated map linking files to relational records

How we measure success

  • Share of files with a unique identifier and complete metadata
  • Share of extracted references matched to a validated key
  • Candidate keys confirmed or rejected by engineers
  • Time to find every document related to a record

We report these measures throughout the engagement. Figures are published only with the client’s approval.

Platforms

  • Amazon S3
  • Azure Blob Storage
  • Google Cloud Storage
  • MongoDB
  • Vector databases

02Case study

Data Quality Assurance

Rules, automated checks and reconciliation that stop bad data before it reaches a report or a new system.

  • Business analysis
  • Data migration
Source systemsQuality rulesTrusted dataSet aside for review
Discuss a similar project

The problem

When reports disagree, the cause is rarely the report. Duplicate customers, missing values, inconsistent formats and broken references move quietly between systems and surface only when someone questions a number.

In a migration, the same problems are copied into the new platform unless they are measured and corrected first.

Our approach

  1. Profile the data

    Measure completeness, uniqueness, formats and value ranges in each critical table to set a baseline before anything changes.

  2. Agree on quality rules

    With the people who own each data domain, define rules for the fields that matter most, covering completeness, validity, uniqueness, consistency and timeliness, each with a threshold.

  3. Automate the checks

    The rules run inside the data pipelines on every load. Records that fail are set aside with the reason, never silently dropped or loaded.

  4. Reconcile source and target

    Row counts, control totals and record-level comparisons show that what arrived matches what left. This is how every load and every migration is signed off.

  5. Report and fix at the source

    A scorecard tracks quality by domain over time, and each recurring issue is traced to the process or system that creates it, with an owner to fix it.

What you receive

  • Data profiling baseline
  • Catalog of quality rules, with owners and thresholds
  • Automated validation in the pipelines, with failing records set aside for review
  • Reconciliation report for each load or migration cycle
  • Data quality scorecard and remediation log

How we measure success

  • Rule pass rate for each critical data element
  • Duplicate and incomplete record rates
  • Differences between source and target after reconciliation
  • Time from detecting an issue to resolving it

We report these measures throughout the engagement. Figures are published only with the client’s approval.

Platforms

  • SQL
  • dbt
  • Apache Airflow
  • Power BI

03Case study

Minimalist Data Storage Cost

Pay only for the data you need, on the storage it needs, without losing the history your reporting depends on.

  • Software architecture
  • Business analysis
Data in daily useLower-cost tierArchiveDuplicate copy removed
Discuss a similar project

The problem

Storage bills tend to grow faster than the value of what is stored. Copies of the same data pile up across systems, backups and logs are kept indefinitely, and everything sits on the most expensive tier because nobody is sure what can be moved.

Deleting the wrong thing would be worse, so nothing changes, and the bill keeps growing.

Our approach

  1. Inventory and classify

    Map every data store with its size, owner, growth, how often it is read and any legal or contractual retention requirement.

  2. Remove duplication

    Find redundant copies and extracts, agree on one authoritative source for each dataset, and retire the rest once they are verified as redundant.

  3. Match storage to use

    Lifecycle rules move data that is rarely read to lower-cost storage tiers and archives automatically, while data in daily use stays fast.

  4. Store efficiently

    Compressed, columnar formats for analytical history, and databases and backups sized to what they actually hold.

  5. Keep history on purpose

    Retention rules remove what is no longer required and keep what is, so the historical records behind your reporting and compliance are preserved.

  6. Make cost visible

    Every dataset is tagged with an owner and a purpose, with budgets and alerts so storage costs do not drift back up.

What you receive

  • Storage inventory and cost baseline
  • Data classification and retention policy
  • Lifecycle and tiering rules
  • Plan for removing duplicate data, verified before anything is deleted
  • Cost allocation tags and a monthly cost report

How we measure success

  • Monthly storage cost, by owner and purpose
  • Share of data stored on the tier that matches how often it is used
  • Redundant copies retired
  • Historical records preserved and verified

We report these measures throughout the engagement. Figures are published only with the client’s approval.

Platforms

  • Amazon S3 storage classes
  • Azure Blob Storage access tiers
  • Google Cloud Storage classes
  • Apache Parquet

See How We Modernize Systems Like Yours

Every enterprise data stack carries unique legacy constraints. Schedule a 20-minute architecture review where our lead engineers will break down anonymized migration blueprints, schema mapping strategies, and ROI metrics tailored to your setup.

Schedule an architecture review