Home / Stacks / Lakehouse & Big Data Stack
🏔️

Lakehouse & Big Data Stack

Store and crunch petabyte-scale data on open formats — Apache Spark and Dask for distributed compute, Iceberg table format, Trino federated queries, BigQuery warehouse, and Airbyte ingestion

6 skills · Works with Claude Code, Codex, Cursor & more

📊 Data
RARE

Apache Spark & PySpark

Process big data at scale with Apache Spark. Writes PySpark ETL jobs, builds DataFrame and Spark SQL transformations, tunes shuffles, partitions, joins, and caching, fixes data skew, and runs on Databricks, EMR, Dataproc, or Kubernetes.

Community 2.6K
Scanned
spark pyspark big-data
mkdir -p ~/.claude/skills/apache-spark && curl -fsSL https://raw.githubusercontent.com/TerminalSkills/skills/main/skills/apache-spark/SKILL.md -o ~/.claude/skills/apache-spark/SKILL.md
📊 Data
RARE

Apache Iceberg Lakehouse

Build an open lakehouse on Apache Iceberg. Designs Iceberg tables and partitions, manages schema and partition evolution, handles snapshots, time travel, and compaction, and queries from Spark, Trino, and DuckDB across S3 or GCS.

Community 1.6K
Scanned
iceberg lakehouse table-format
mkdir -p ~/.claude/skills/apache-iceberg && curl -fsSL https://raw.githubusercontent.com/Evan-Kim2028/agent-skills/main/skills/data/apache-lakehouse/SKILL.md -o ~/.claude/skills/apache-iceberg/SKILL.md
📊 Data
RARE

Trino Query Optimizer

Analyze and optimize Trino queries. Reads EXPLAIN plans, predicts CPU, memory, and scan cost before running, rewrites expensive federated queries, and surfaces partition-pruning and join-order fixes for faster, cheaper distributed SQL.

Community 1.2K
Scanned
trino presto query-optimization
mkdir -p ~/.claude/skills/trino-query-engine && curl -fsSL https://raw.githubusercontent.com/razorpay/trino-gateway/master/.agents/skills/trino-analyzer/SKILL.md -o ~/.claude/skills/trino-query-engine/SKILL.md
📊 Data
RARE

Google BigQuery

Query and model data in BigQuery. Writes performant GoogleSQL, partitions and clusters tables to cut scan costs, builds scheduled queries and materialized views, and manages datasets, UDFs, and BI Engine for fast, cheap analytics.

Community 2.5K
Scanned
bigquery data-warehouse sql
mkdir -p ~/.claude/skills/bigquery-warehouse && curl -fsSL https://raw.githubusercontent.com/TerminalSkills/skills/main/skills/gcp-bigquery/SKILL.md -o ~/.claude/skills/bigquery-warehouse/SKILL.md
📊 Data
RARE

Dask Parallel Python

Scale Python and pandas with Dask. Parallelizes DataFrame and array workloads across cores or a cluster, tunes partitions and the task graph, handles larger-than-memory data, and diagnoses spill and shuffle bottlenecks from the dashboard.

Community 1.4K
Scanned
dask parallel pandas
mkdir -p ~/.claude/skills/dask-parallel && curl -fsSL https://raw.githubusercontent.com/K-Dense-AI/scientific-agent-skills/main/skills/dask/SKILL.md -o ~/.claude/skills/dask-parallel/SKILL.md
📊 Data
RARE

Airbyte Data Integration

Move data with Airbyte ELT. Configures sources and destinations, builds and tests custom connectors with the CDK, sets up incremental and CDC replication, schedules syncs, and wires failure alerting across 300+ APIs and databases.

Community 2.1K
Scanned
airbyte elt data-integration
mkdir -p ~/.claude/skills/airbyte-connectors && curl -fsSL https://raw.githubusercontent.com/TerminalSkills/skills/main/skills/airbyte/SKILL.md -o ~/.claude/skills/airbyte-connectors/SKILL.md

More Stacks

🤝

Agent Engineering Kit

Engineer reliable AI agents — build on the Claude Agent SDK, coordinate teams with multi-agent patterns, give them memory, and keep them safe with mechanical guardrails and evaluation
🕹️

AI Agent Frameworks

Pick the right framework and ship agents fast — type-safe Python with Pydantic AI, multi-agent orchestration with OpenAI Agents SDK, Agno, and CrewAI, TypeScript graphs with LangGraph.js, and typed workflows with Mastra
🤖

AI Builder Stack

Build AI-powered applications with RAG, multi-agent systems, prompt engineering, and MCP integrations

View all stacks →

Added to wishlist