description: The key concepts that make up PostgresML.

* [Getting Started](introduction/getting-started/README.md)

* [Create your database](introduction/getting-started/create-your-database.md)

* [Connect your app](introduction/getting-started/connect-your-app.md)

* [Import your data](introduction/getting-started/import-your-data/README.md)

* [CSV](introduction/getting-started/import-your-data/csv.md)

* [Foreign Data Wrapper](introduction/getting-started/import-your-data/foreign-data-wrapper.md)

* [Machine Learning](introduction/machine-learning/README.md)

* [Natural Language Processing](introduction/machine-learning/natural-language-processing/README.md)

* [Embeddings](introduction/machine-learning/natural-language-processing/embeddings.md)

* [Fill Mask](introduction/machine-learning/natural-language-processing/fill-mask.md)

* [Question Answering](introduction/machine-learning/natural-language-processing/question-answering.md)

* [Summarization](introduction/machine-learning/natural-language-processing/summarization.md)

* [Text Classification](introduction/machine-learning/natural-language-processing/text-classification.md)

* [Text Generation](introduction/machine-learning/natural-language-processing/text-generation.md)

* [Text-to-Text Generation](introduction/machine-learning/natural-language-processing/text-to-text-generation.md)

* [Token Classification](introduction/machine-learning/natural-language-processing/token-classification.md)

* [Translation](introduction/machine-learning/natural-language-processing/translation.md)

* [Zero-shot Classification](introduction/machine-learning/natural-language-processing/zero-shot-classification.md)

* [Supervised Learning](introduction/machine-learning/supervised-learning/README.md)

* [Data Pre-processing](introduction/machine-learning/supervised-learning/data-pre-processing.md)

* [Regression](introduction/machine-learning/supervised-learning/regression.md)

* [Classification](introduction/machine-learning/supervised-learning/classification.md)

* [Hyperparameter Search](introduction/machine-learning/supervised-learning/hyperparameter-search.md)

* [Joint Optimization](introduction/machine-learning/supervised-learning/joint-optimization.md)

* [Unsupervised Learning](introduction/machine-learning/unsupervised-learning.md)

* [SDKs](introduction/machine-learning/sdks/README.md)

* [Overview](introduction/machine-learning/sdks/overview.md)

* [Getting Started](introduction/machine-learning/sdks/getting-started.md)

* [OpenSourceAI](introduction/machine-learning/sdks/opensourceai.md)

* [Collections](introduction/machine-learning/sdks/collections.md)

* [Pipelines](introduction/machine-learning/sdks/pipelines.md)

* [Search](introduction/machine-learning/sdks/search.md)

* [Tutorials](introduction/machine-learning/sdks/tutorials/README.md)

* [Semantic Search](introduction/machine-learning/sdks/tutorials/semantic-search.md)

* [Semantic Search using Instructor model](introduction/machine-learning/sdks/tutorials/semantic-search-using-instructor-model.md)

* [Extractive Question Answering](introduction/machine-learning/sdks/tutorials/extractive-question-answering.md)

* [Summarizing Question Answering](introduction/machine-learning/sdks/tutorials/summarizing-question-answering.md)

* [Cloud Database](product/cloud-database/README.md)

* [Serverless databases](product/cloud-database/serverless-databases.md)

* [Dedicated](product/cloud-database/dedicated.md)

* [Enterprise](product/cloud-database/plans.md)

* [Vector Database](product/vector-database.md)

* [PgCat Proxy](product/pgcat/README.md)

* [Features](product/pgcat/features.md)

* [Installation](product/pgcat/installation.md)

* [Configuration](product/pgcat/configuration.md)

* [Chatbots](use-cases/chatbots.md)

* [Search](use-cases/improve-search-results-with-machine-learning.md)

* [Embeddings](use-cases/embeddings/README.md)

* [Generating LLM embeddings with open source models in PostgresML](use-cases/embeddings/generating-llm-embeddings-with-open-source-models-in-postgresml/README.md)

* [Tuning vector recall while generating query embeddings in the database](use-cases/embeddings/generating-llm-embeddings-with-open-source-models-in-postgresml/tuning-vector-recall-while-generating-query-embeddings-in-the-database.md)

* [Personalize embedding results with application data in your database](use-cases/embeddings/personalize-embedding-results-with-application-data-in-your-database.md)

* [Time-series Forecasting](use-cases/time-series-forecasting.md)

* [Fraud Detection](use-cases/fraud-detection.md)

* [Recommendation Engine](use-cases/recommendation-engine.md)

* [FAQs](resources/faqs.md)

* [Data Storage & Retrieval](resources/data-storage-and-retrieval/README.md)

* [Tabular data](resources/data-storage-and-retrieval/tabular-data.md)

* [Documents](resources/data-storage-and-retrieval/documents.md)

* [Partitioning](resources/data-storage-and-retrieval/partitioning.md)

* [LLM based pipelines with PostgresML and dbt (data build tool)](resources/data-storage-and-retrieval/llm-based-pipelines-with-postgresml-and-dbt-data-build-tool.md)

* [Benchmarks](resources/benchmarks/README.md)

* [PostgresML is 8-40x faster than Python HTTP microservices](resources/benchmarks/postgresml-is-8-40x-faster-than-python-http-microservices.md)

* [Million Requests per Second](resources/benchmarks/million-requests-per-second.md)

* [MindsDB vs PostgresML](resources/benchmarks/mindsdb-vs-postgresml.md)

* [GGML Quantized LLM support for Huggingface Transformers](resources/benchmarks/ggml-quantized-llm-support-for-huggingface-transformers.md)

* [Making Postgres 30 Percent Faster in Production](resources/benchmarks/making-postgres-30-percent-faster-in-production.md)

* [Developer Docs](resources/developer-docs/README.md)

* [Local Docker Development](resources/developer-docs/quick-start-with-docker.md)

* [Installation](resources/developer-docs/installation.md)

* [Contributing](resources/developer-docs/contributing.md)

* [Distributed Training](resources/developer-docs/distributed-training.md)

* [GPU Support](resources/developer-docs/gpu-support.md)

* [Deploying PostgresML](resources/developer-docs/deploying-postgresml/README.md)

* [Monitoring](resources/developer-docs/deploying-postgresml/monitoring.md)

* [Self-hosting](resources/developer-docs/self-hosting/README.md)

* [Pooler](resources/developer-docs/self-hosting/pooler.md)

* [Building from source](resources/developer-docs/self-hosting/building-from-source.md)

* [Replication](resources/developer-docs/self-hosting/replication.md)

* [Backups](resources/developer-docs/self-hosting/backups.md)

* [Running on EC2](resources/developer-docs/self-hosting/running-on-ec2.md)

<figure><img src="../../.gitbook/assets/architecture.png" alt=""><figcaption></figcaption></figure>

<figure><img src="../../.gitbook/assets/image (6).png" alt="Sign up" width="356"><figcaption></figcaption></figure>

<figure><img src="../../.gitbook/assets/image (7).png" alt=""><figcaption></figcaption></figure>

<figure><img src="../../.gitbook/assets/Screenshot from 2023-11-27 23-21-36.png" alt=""><figcaption></figcaption></figure>

Machine learning always depends on input data, whether it's generating text with pretrained LLMs, training a retention model on customer data, or predicting session abandonment in real time. Just like any PostgreSQL database, PostgresML can be configured as the authoritative application data store, a streaming replica from some other primary, or use foreign data wrappers to query another data host on demand. Depending on how frequently your data changes and where your authoritative data resides, different methodologies imply different tradeoffs.

PostgresML can easily ingest data from your existing data stores.&#x20;

Data that changes infrequently can be easily imported into PostgresML using `COPY`. All you have to do is export your data as a CSV file, create a table in Postgres to store it, and import it using the command line.

{% content-ref url="csv.md" %}

[csv.md](csv.md)

{% endcontent-ref %}

Importing data from online databases can be done with foreign data wrappers. Hosted PostgresML databases come with both `postgres_fdw` and `dblink` extensions pre-installed, so you can import data from any of your existing Postgres databases, and export machine learning artifacts from PostgresML using just a few lines of SQL.

{% content-ref url="foreign-data-wrapper.md" %}

[foreign-data-wrapper.md](foreign-data-wrapper.md)

{% endcontent-ref %}

Data that changes infrequently can be easily imported into PostgresML using `COPY`. All you have to do is export your data as a CSV file, create a table in Postgres to store it, and import it using the command line.

Let's use a simple CSV file with 3 columns as an example:

| Column | Data type | Example |

| ---------------- | --------- | ------- |

| name | text | John |

| age | integer | 30 |

| is\_paying\_user | boolean | true |

If you're using a Postgres database already, you can export any table as CSV with just one command:

psql -c "\copy your_table TO '~/Desktop/your_table.csv' CSV HEADER"

If you're using another data store, it should almost always provide a CSV export functionality, since CSV is the most commonly used data format in machine learning.

Creating a table in Postgres with the correct schema is as easy as:

Once you have a table and your data exported as CSV, importing it can also be done with just one command:

psql -c "\copy your_table FROM '~/Desktop/your_table.csv' CSV HEADER"

We took our export command and changed `TO` to `FROM`, and that's it. Make sure you're connecting to your PostgresML database when importing data.

If your data changed, repeat this process again. To avoid duplicate entries in your table, you can truncate (or delete) all rows beforehand: