Browse tools in the Data Pipelines space.
191 tools
Airbyte is an open-source tool designed for data integration, enabling the movement of data from APIs, databases, and fi...
Akshare is an open-source Python library designed to provide a simple and efficient interface for accessing a wide range...
Apache Airflow is an open-source platform designed to programmatically author, schedule, and monitor workflows. It is wr...
Apache Flink is an open-source framework for distributed stream and batch data processing. It provides high throughput, ...
Spark is a powerful open-source framework designed to handle big data with speed and efficiency. It supports various dat...
Dagster is a powerful open-source framework that helps developers build, deploy, and maintain robust data pipelines. It ...
dbt (data build tool) is the standard for data transformation in modern data stacks. It enables analysts and engineers t...
Fastai is a high-level wrapper built on top of Pytorch that simplifies the process of building deep learning models. It ...
Apache Hadoop is a distributed computing framework that supports the processing of large data sets in a distributed envi...
Kubeflow simplifies the deployment of machine learning workflows on Kubernetes, making them portable and scalable. It pr...
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflo...
Metaflow is a Python-based open-source framework that simplifies the process of building and deploying machine learning ...
OpenBB is a comprehensive financial data platform designed to support analysts, quantitative researchers, and artificial...
Pandas provides easy-to-use data structures and data analysis tools for Python. It is essential for data manipulation, c...
Pathway is a real-time data processing framework that enables developers to build applications using reactive dataflows ...
Prefect is a modern workflow orchestration framework that simplifies building, scheduling, and monitoring complex data p...
PyTorch Lightning is a lightweight wrapper that makes it easier to train deep learning models with PyTorch. It simplifie...
Qlib is an AI-oriented Quant investment platform that uses advanced machine learning techniques to support quantitative ...
Scrapy is a powerful Python-based open-source web crawling framework used for extracting data from websites. It's design...
Unstructured is an open-source ETL solution that transforms complex documents into clean, structured formats. It's ideal...
Apache Atlas is an open-source tool providing metadata management and data governance capabilities across the Hadoop eco...
aws-sdk-pandas is a library that extends the capabilities of Pandas to work seamlessly with AWS services, enabling devel...
CSVKit is a powerful set of command-line tools that makes it easy to convert data into CSV format, manipulate CSV files,...
Dask is a flexible parallel computing library designed to scale from single machines to large clusters. It integrates wi...
DataFramesMeta.jl provides metaprogramming tools to work with DataFrames in the Julia programming language, enhancing da...
Databricks is the lakehouse platform (built on Apache Spark/Delta Lake) unifying data engineering, warehousing, and mach...
A library to compare data frames from Pandas, Polars, and Spark. It provides detailed statistics and allows users to adj...
DataDeps.jl is a Julia package that simplifies the process of setting up and managing datasets in scientific computing p...
DataFrames.jl is a powerful and flexible Julia package designed to work with tabular data, providing tools for data mani...
Datatrove simplifies data processing by offering a set of customizable pipeline processing blocks, freeing users from co...
Showing 30 of 191
