Dataplugins
Dataplugins simply define a source of data from a given repository. Meltano Cloud provides a number of pre-configured platform-wide dataplugins out-the-box, as well as the ability to create custom dataplugins through the API. From these, pipeline jobs can be run to inject data into a workspace.
Objects
Dataplugin
| Path | JSON Type | Format | Description |
|---|---|---|---|
id | string | Version 4 UUID | The dataplugin ID |
name | string | The dataplugin name | |
description | string | A description of the dataplugin | |
repositoryUrl | string | URL | The dataplugin repository URL |
settings | object[] | Array of Settings | The dataplugin settings |
{
"id" : "32374218-07a4-4a49-92f6-476339ae7b31",
"pluginType" : "LOADER",
"name" : "target-postgres",
"namespace" : "postgres_transferwise",
"variant" : "matatika",
"label" : "Postgres Warehouse",
"description" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.",
"logoUrl" : "/assets/logos/loaders/postgres.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/target-postgres/",
"pipUrl" : "git+https://github.com/Matatika/pipelinewise-target-postgres.git@v0.2.1",
"repo" : "https://github.com/Matatika/pipelinewise-target-postgres",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "user",
"aliases" : [ "username" ],
"label" : "User",
"options" : [ ],
"kind" : "STRING",
"description" : "The username used to connect to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "password",
"aliases" : [ ],
"label" : "Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password used to authenticate the user.",
"hidden" : false,
"sensitive" : true,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "host",
"aliases" : [ "address" ],
"label" : "Host",
"options" : [ ],
"kind" : "STRING",
"description" : "The hostname or IP address of the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "port",
"aliases" : [ ],
"label" : "Port",
"value" : "5432",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The port number used to connect to the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "dbname",
"aliases" : [ "database" ],
"label" : "Database Name",
"options" : [ ],
"kind" : "STRING",
"description" : "The name of the database to connect to.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "default_target_schema",
"aliases" : [ ],
"label" : "Default Target Schema",
"value" : "analytics",
"options" : [ ],
"kind" : "STRING",
"description" : "The default schema to use when writing data to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "state_schema",
"aliases" : [ ],
"label" : "State Schema",
"value" : "public",
"options" : [ ],
"kind" : "STRING",
"description" : "The default schema to use when writing Meltano state.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "ssl",
"aliases" : [ ],
"label" : "SSL",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to use SSL encryption when connecting to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false,
"value_post_processor" : "STRINGIFY"
}, {
"name" : "batch_size_rows",
"aliases" : [ ],
"label" : "Batch Size Rows",
"value" : "100000",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of rows to write to the Postgres Warehouse in each batch.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "underscore_camel_case_fields",
"aliases" : [ ],
"label" : "Underscore Camel Case Fields",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to convert field names from camel case to underscore-separated format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flush_all_streams",
"aliases" : [ ],
"label" : "Flush All Streams",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to flush all streams to the Postgres Warehouse before closing the connection.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "parallelism",
"aliases" : [ ],
"label" : "Parallelism",
"value" : "-1",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "parallelism_max",
"aliases" : [ ],
"label" : "Max Parallelism",
"value" : "16",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "default_target_schema_select_permission",
"aliases" : [ ],
"label" : "Default Target Schema Select Permission",
"options" : [ ],
"kind" : "STRING",
"description" : "The permission level required to select data from the default target schema.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "schema_mapping",
"aliases" : [ ],
"label" : "Schema Mapping",
"options" : [ ],
"kind" : "STRING",
"description" : "A mapping of source schema names to target schema names.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "add_metadata_columns",
"aliases" : [ ],
"label" : "Add Metadata Columns",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to add metadata columns to the target table.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "hard_delete",
"aliases" : [ ],
"label" : "Hard Delete",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to perform hard deletes when deleting data from the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "data_flattening_max_level",
"aliases" : [ ],
"label" : "Data Flattening Max Level",
"value" : "10",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum level of nested data structures to flatten when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "primary_key_required",
"aliases" : [ ],
"label" : "Primary Key Required",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not a primary key is required for the target table.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "validate_records",
"aliases" : [ ],
"label" : "Validate Records",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to validate records before writing them to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "temp_dir",
"aliases" : [ ],
"label" : "Temporary Directory",
"options" : [ ],
"kind" : "STRING",
"description" : "The directory to use for temporary files when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.host",
"aliases" : [ ],
"label" : "SSH Tunnel Host",
"options" : [ ],
"kind" : "STRING",
"description" : "Host of the bastion host, this is the host we'll connect to via ssh",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.port",
"aliases" : [ ],
"label" : "SSH Tunnel Port",
"value" : "22",
"options" : [ ],
"kind" : "INTEGER",
"description" : "Port to connect to bastion host",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.private_key",
"aliases" : [ ],
"label" : "SSH Tunnel Private Key",
"options" : [ ],
"kind" : "PASSWORD",
"description" : "A base64 encoded Private Key for authentication to the bastion host w/ key pair auth",
"hidden" : false,
"sensitive" : true,
"encoding" : "BASE64",
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.private_key_password",
"aliases" : [ ],
"label" : "SSH Tunnel Private Key Password",
"options" : [ ],
"kind" : "PASSWORD",
"description" : "Private Key Password, leave None if no password is set",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.username",
"aliases" : [ ],
"label" : "SSH Tunnel Username",
"options" : [ ],
"kind" : "STRING",
"description" : "Username to connect to bastion host",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.password",
"aliases" : [ ],
"label" : "SSH Tunnel Password",
"options" : [ ],
"kind" : "STRING",
"description" : "Password to connect to bastion host w/ basic auth",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.host_setting_name",
"aliases" : [ ],
"value" : "host",
"options" : [ ],
"kind" : "STRING",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.port_setting_name",
"aliases" : [ ],
"value" : "port",
"options" : [ ],
"kind" : "STRING",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.\n\n## Settings\n\n\n### User\n\nThe username used to connect to the Postgres Warehouse.\n\n### Password\n\nThe password used to authenticate the user.\n\n### Host\n\nThe hostname or IP address of the Postgres Warehouse server.\n\n### Port\n\nThe port number used to connect to the Postgres Warehouse server.\n\n### Database Name\n\nThe name of the database to connect to.\n\n### Default Target Schema\n\nThe default schema to use when writing data to the Postgres Warehouse.\n\n### State Schema\n\nThe default schema to use when writing Meltano state.\n\n### Batch Size Rows\n\nThe number of rows to write to the Postgres Warehouse in each batch.\n\n### Underscore Camel Case Fields\n\nWhether or not to convert field names from camel case to underscore-separated format.\n\n### Primary Key Required\n\nWhether or not a primary key is required for the target table.\n\n### Validate Records\n\nWhether or not to validate records before writing them to the Postgres Warehouse.\n\n### SSH Tunnel Host\n\nHost of the bastion host, this is the host we'll connect to via ssh\n\n### SSH Tunnel Port\n\nPort to connect to bastion host\n\n### SSH Tunnel Private Key\n\nA base64 encoded Private Key for authentication to the bastion host w/ key pair auth\n\n### SSH Tunnel Private Key Password\n\nPrivate Key Password, leave None if no password is set\n\n### SSH Tunnel Username\n\nUsername to connect to bastion host\n\n### SSH Tunnel Password\n\nPassword to connect to bastion host w/ basic auth",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31",
"type" : "DELETE"
}
}
}
Setting
| Path | JSON Type | Format | Description |
|---|---|---|---|
name | string | The setting name | |
value | string | The setting default value | |
label | string | The setting label | |
protected | bool | The setting protection status | |
kind | string | Setting Kind | The setting kind |
description | string | A description of the setting | |
placeholder | string | The setting placeholder text | |
envAliases | string[] | Environment variable aliases for the setting | |
documentation | string | URL | The setting documentation URL |
oauth | OAuth | The setting OAuth configuration | |
env | string |
OAuth
| Path | JSON Type | Format | Description |
|---|---|---|---|
provider | string | The OAuth provider |
Formats
Setting Kind
string
| Value | Description |
|---|---|
STRING | String setting |
INTEGER | Integer setting |
PASSWORD | Password setting |
HIDDEN | Hidden setting |
BOOLEAN | Boolean setting |
DATE_ISO8601 | ISO 8601 date setting |
EMAIL | Email setting |
OAUTH | OAuth setting |
FILE | File setting |
ARRAY | Array setting |
Requests
View all supported dataplugins
GET /api/dataplugins
Returns all dataplugins supported by Meltano Cloud.
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://staging.meltano.com:443/api/dataplugins' -i -X GET
import requests
url = "https://staging.meltano.com:443/api/dataplugins"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Dataplugin collection with HAL links.
{
"_embedded" : {
"dataplugins" : [ {
"id" : "99534d28-7277-4da2-af01-e7da92688d34",
"pluginType" : "EXTRACTOR",
"name" : "singer-tap-zenhub",
"namespace" : "singer_tap_zenhub",
"variant" : "chillu",
"label" : "ZenHub",
"description" : "ZenHub is a project management tool for software development teams that integrates with GitHub.\n\nZenHub is a powerful project management tool that helps software development teams manage their projects more efficiently by integrating with GitHub. With ZenHub, teams can visualize their workflow, plan and prioritize tasks, and collaborate seamlessly within their existing GitHub environment. The tool offers features such as burndown charts, epics, and release reports to help teams stay on track and deliver high-quality software on time. ZenHub also provides real-time insights into project progress, enabling teams to make data-driven decisions and continuously improve their processes.",
"logoUrl" : "/assets/logos/extractors/zenhub.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/singer-tap-zenhub/",
"pipUrl" : "git+https://github.com/chillu/singer-tap-zenhub.git",
"repo" : "https://github.com/chillu/singer-tap-zenhub",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "ZenHub is a project management tool for software development teams that integrates with GitHub.\n\nZenHub is a powerful project management tool that helps software development teams manage their projects more efficiently by integrating with GitHub. With ZenHub, teams can visualize their workflow, plan and prioritize tasks, and collaborate seamlessly within their existing GitHub environment. The tool offers features such as burndown charts, epics, and release reports to help teams stay on track and deliver high-quality software on time. ZenHub also provides real-time insights into project progress, enabling teams to make data-driven decisions and continuously improve their processes.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/99534d28-7277-4da2-af01-e7da92688d34"
}
}
}, {
"id" : "a79d71dc-024a-4db5-9f1c-8f3f0a6aad06",
"pluginType" : "EXTRACTOR",
"name" : "tap-3plcentral",
"namespace" : "tap_3plcentral",
"variant" : "singer-io",
"label" : "3PLCentral",
"description" : "3PLCentral is a cloud-based warehouse management system designed for third-party logistics providers.\n\n3PLCentral provides a comprehensive platform for managing warehouse operations, including inventory management, order fulfillment, shipping and receiving, and reporting. The system is designed specifically for third-party logistics providers, allowing them to manage multiple clients and warehouses from a single platform. With 3PLCentral, users can automate many of their warehouse processes, reducing errors and improving efficiency. The system also integrates with a variety of other software tools, including e-commerce platforms, shipping carriers, and accounting software, making it a powerful tool for managing all aspects of a logistics operation.",
"logoUrl" : "/assets/logos/extractors/3plcentral.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-3plcentral/",
"pipUrl" : "tap-3plcentral",
"repo" : "https://github.com/singer-io/tap-3plcentral",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "client_id",
"aliases" : [ ],
"label" : "Client Id",
"options" : [ ],
"kind" : "STRING",
"description" : "The unique identifier for the client accessing the API.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The date from which to start retrieving data.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "base_url",
"aliases" : [ ],
"label" : "Base Url",
"options" : [ ],
"kind" : "STRING",
"description" : "The base URL for the 3PLCentral API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "tpl_key",
"aliases" : [ ],
"label" : "TPL Key",
"options" : [ ],
"kind" : "STRING",
"description" : "The API key provided by 3PLCentral.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "user_agent",
"aliases" : [ ],
"label" : "User Agent",
"options" : [ ],
"kind" : "STRING",
"description" : "The user agent string to identify the client application.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "customer_id",
"aliases" : [ ],
"label" : "Customer Id",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The unique identifier for the customer associated with the API request.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "client_secret",
"aliases" : [ ],
"label" : "Client Secret",
"options" : [ ],
"kind" : "STRING",
"description" : "The secret key used for authentication.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "user_login_id",
"aliases" : [ ],
"label" : "User Login Id",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The unique identifier for the user associated with the API request.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "facility_id",
"aliases" : [ ],
"label" : "Facility Id",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The unique identifier for the facility associated with the API request.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "3PLCentral is a cloud-based warehouse management system designed for third-party logistics providers.\n\n3PLCentral provides a comprehensive platform for managing warehouse operations, including inventory management, order fulfillment, shipping and receiving, and reporting. The system is designed specifically for third-party logistics providers, allowing them to manage multiple clients and warehouses from a single platform. With 3PLCentral, users can automate many of their warehouse processes, reducing errors and improving efficiency. The system also integrates with a variety of other software tools, including e-commerce platforms, shipping carriers, and accounting software, making it a powerful tool for managing all aspects of a logistics operation.\n\n## Settings\n\n\n### Client Id\n\nThe unique identifier for the client accessing the API.\n\n### Start Date\n\nThe date from which to start retrieving data.\n\n### Base Url\n\nThe base URL for the 3PLCentral API.\n\n### TPL Key\n\nThe API key provided by 3PLCentral.\n\n### User Agent\n\nThe user agent string to identify the client application.\n\n### Customer Id\n\nThe unique identifier for the customer associated with the API request.\n\n### Client Secret\n\nThe secret key used for authentication.\n\n### User Login Id\n\nThe unique identifier for the user associated with the API request.\n\n### Facility Id\n\nThe unique identifier for the facility associated with the API request.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/a79d71dc-024a-4db5-9f1c-8f3f0a6aad06"
}
}
}, {
"id" : "546921a5-6202-4440-87e4-396418f901c1",
"pluginType" : "EXTRACTOR",
"name" : "tap-abcfinancial",
"namespace" : "tap_abcfinancial",
"variant" : "dmzobel",
"label" : "ABC Fitness Solutions",
"description" : "ABC Fitness Solutions: A software solution for fitness businesses to manage their operations and engage with their clients.\n\nABC Fitness Solutions is a comprehensive software solution designed for fitness businesses to manage their operations and engage with their clients. The platform offers features such as membership management, scheduling, billing, and reporting, as well as tools for client communication and engagement, such as mobile apps and social media integration. With ABC Fitness Solutions, fitness businesses can streamline their operations, improve client retention, and grow their business.",
"logoUrl" : "/assets/logos/extractors/abcfinancial.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-abcfinancial/",
"pipUrl" : "git+https://github.com/dmzobel/tap-abcfinancial.git",
"repo" : "https://github.com/dmzobel/tap-abcfinancial",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "ABC Fitness Solutions: A software solution for fitness businesses to manage their operations and engage with their clients.\n\nABC Fitness Solutions is a comprehensive software solution designed for fitness businesses to manage their operations and engage with their clients. The platform offers features such as membership management, scheduling, billing, and reporting, as well as tools for client communication and engagement, such as mobile apps and social media integration. With ABC Fitness Solutions, fitness businesses can streamline their operations, improve client retention, and grow their business.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/546921a5-6202-4440-87e4-396418f901c1"
}
}
}, {
"id" : "8086d6e9-e4c2-4ea0-aede-2d4beee1b1d6",
"pluginType" : "EXTRACTOR",
"name" : "tap-abicart",
"namespace" : "tap_abicart",
"variant" : "hotgluexyz",
"label" : "Abicart",
"description" : "Abicart is an e-commerce platform for businesses to create and manage their online stores.\n\nAbicart is an all-in-one e-commerce platform that allows businesses to easily create and manage their online stores. With features such as customizable templates, product management, payment processing, and marketing tools, Abicart provides a user-friendly solution for businesses of all sizes to sell their products online. The platform also offers integrations with popular third-party services such as Google Analytics, Mailchimp, and Facebook, allowing businesses to streamline their online operations and reach a wider audience.",
"logoUrl" : "/assets/logos/extractors/abicart.svg",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-abicart/",
"pipUrl" : "git+https://gitlab.com/hotglue/tap-abicart.git",
"repo" : "https://gitlab.com/hotglue/tap-abicart",
"executable" : "tap-abicart",
"capabilities" : [ "CATALOG", "STREAM_MAPS", "DISCOVER", "STATE", "ABOUT", "SCHEMA_FLATTENING" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "api_url",
"aliases" : [ ],
"label" : "API URL",
"value" : "https://admin.abicart.se/backend/jsonrpc/v1/",
"options" : [ ],
"kind" : "STRING",
"description" : "The URL endpoint for the Abicart API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_enabled",
"aliases" : [ ],
"label" : "Flattening Enabled",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "A boolean value indicating whether or not to flatten the response data.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_max_depth",
"aliases" : [ ],
"label" : "Flattening Max Depth",
"options" : [ ],
"kind" : "INTEGER",
"description" : "An integer value indicating the maximum depth to flatten the response data.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "password",
"aliases" : [ ],
"label" : "Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password for the Abicart API user.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "stream_map_config",
"aliases" : [ ],
"label" : "Stream Map Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "A configuration object for mapping data streams.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_maps",
"aliases" : [ ],
"label" : "Stream Maps",
"options" : [ ],
"kind" : "OBJECT",
"description" : "An array of data stream maps.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "username",
"aliases" : [ ],
"label" : "Username",
"options" : [ ],
"kind" : "STRING",
"description" : "The username for the Abicart API user.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "webshop",
"aliases" : [ ],
"label" : "Webshop",
"options" : [ ],
"kind" : "STRING",
"description" : "The name or ID of the Abicart webshop to connect to.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Abicart is an e-commerce platform for businesses to create and manage their online stores.\n\nAbicart is an all-in-one e-commerce platform that allows businesses to easily create and manage their online stores. With features such as customizable templates, product management, payment processing, and marketing tools, Abicart provides a user-friendly solution for businesses of all sizes to sell their products online. The platform also offers integrations with popular third-party services such as Google Analytics, Mailchimp, and Facebook, allowing businesses to streamline their online operations and reach a wider audience.\n\n## Settings\n\n\n### API URL\n\nThe URL endpoint for the Abicart API.\n\n### Flattening Enabled\n\nA boolean value indicating whether or not to flatten the response data.\n\n### Flattening Max Depth\n\nAn integer value indicating the maximum depth to flatten the response data.\n\n### Password\n\nThe password for the Abicart API user.\n\n### Stream Map Config\n\nA configuration object for mapping data streams.\n\n### Stream Maps\n\nAn array of data stream maps.\n\n### Username\n\nThe username for the Abicart API user.\n\n### Webshop\n\nThe name or ID of the Abicart webshop to connect to.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/8086d6e9-e4c2-4ea0-aede-2d4beee1b1d6"
}
}
}, {
"id" : "687779b9-7ea8-4e3b-b42c-c39853eab9df",
"pluginType" : "EXTRACTOR",
"name" : "tap-abra",
"namespace" : "tap_abra",
"variant" : "hotgluexyz",
"label" : "Abra",
"description" : "Abra is a mobile app that allows users to invest in cryptocurrencies and other assets.\n\nAbra is a cryptocurrency investment app that allows users to buy, sell, and hold various cryptocurrencies, as well as traditional assets like stocks and ETFs, all in one place. The app offers a user-friendly interface and allows users to fund their accounts using bank transfers, credit cards, or cryptocurrencies. Abra also offers a unique feature that allows users to invest in multiple assets at once, using a single wallet. Additionally, Abra offers a built-in exchange that allows users to convert between different cryptocurrencies and fiat currencies. Overall, Abra is a comprehensive investment app that makes it easy for users to invest in a variety of assets, including cryptocurrencies.",
"logoUrl" : "/assets/logos/extractors/abra.svg",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-abra/",
"pipUrl" : "git+https://gitlab.com/hotglue/tap-abra.git",
"repo" : "https://gitlab.com/hotglue/tap-abra",
"executable" : "tap-abra",
"capabilities" : [ "CATALOG", "STREAM_MAPS", "DISCOVER", "STATE", "ABOUT", "SCHEMA_FLATTENING" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "flattening_enabled",
"aliases" : [ ],
"label" : "Flattening Enabled",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "A setting that determines whether or not to flatten nested JSON objects in the API response.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_max_depth",
"aliases" : [ ],
"label" : "Flattening Max Depth",
"options" : [ ],
"kind" : "INTEGER",
"description" : "A setting that determines the maximum depth of nested JSON objects to flatten in the API response.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "password",
"aliases" : [ ],
"label" : "Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password used to authenticate the user when connecting to the Abra API.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The date from which to start streaming data from the Abra API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_map_config",
"aliases" : [ ],
"label" : "Stream Map Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "A configuration file that maps the API response to a desired output format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_maps",
"aliases" : [ ],
"label" : "Stream Maps",
"options" : [ ],
"kind" : "OBJECT",
"description" : "A list of stream maps that define how to map the API response to a desired output format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "username",
"aliases" : [ ],
"label" : "Username",
"options" : [ ],
"kind" : "STRING",
"description" : "The username used to authenticate the user when connecting to the Abra API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Abra is a mobile app that allows users to invest in cryptocurrencies and other assets.\n\nAbra is a cryptocurrency investment app that allows users to buy, sell, and hold various cryptocurrencies, as well as traditional assets like stocks and ETFs, all in one place. The app offers a user-friendly interface and allows users to fund their accounts using bank transfers, credit cards, or cryptocurrencies. Abra also offers a unique feature that allows users to invest in multiple assets at once, using a single wallet. Additionally, Abra offers a built-in exchange that allows users to convert between different cryptocurrencies and fiat currencies. Overall, Abra is a comprehensive investment app that makes it easy for users to invest in a variety of assets, including cryptocurrencies.\n\n## Settings\n\n\n### Flattening Enabled\n\nA setting that determines whether or not to flatten nested JSON objects in the API response.\n\n### Flattening Max Depth\n\nA setting that determines the maximum depth of nested JSON objects to flatten in the API response.\n\n### Password\n\nThe password used to authenticate the user when connecting to the Abra API.\n\n### Start Date\n\nThe date from which to start streaming data from the Abra API.\n\n### Stream Map Config\n\nA configuration file that maps the API response to a desired output format.\n\n### Stream Maps\n\nA list of stream maps that define how to map the API response to a desired output format.\n\n### Username\n\nThe username used to authenticate the user when connecting to the Abra API.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/687779b9-7ea8-4e3b-b42c-c39853eab9df"
}
}
}, {
"id" : "c3eb7f52-9cb7-4501-8ae1-4651fba48937",
"pluginType" : "EXTRACTOR",
"name" : "tap-aconex",
"namespace" : "tap_aconex",
"variant" : "fostersdata",
"label" : "Aconex",
"description" : "Aconex is a cloud-based construction management software.\n\nAconex is a platform that helps construction and engineering companies manage their projects by providing tools for document management, communication, and collaboration. It allows users to store and share project information, track progress, and manage workflows. The software is designed to improve efficiency, reduce risk, and increase transparency across all stakeholders involved in a construction project. Aconex is used by companies in various industries, including infrastructure, energy, mining, and real estate.",
"logoUrl" : "/assets/logos/extractors/aconex.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-aconex/",
"pipUrl" : "git+https://github.com/FostersData/tap-aconex.git",
"repo" : "https://github.com/FostersData/tap-aconex",
"capabilities" : [ "CATALOG", "DISCOVER" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Aconex is a cloud-based construction management software.\n\nAconex is a platform that helps construction and engineering companies manage their projects by providing tools for document management, communication, and collaboration. It allows users to store and share project information, track progress, and manage workflows. The software is designed to improve efficiency, reduce risk, and increase transparency across all stakeholders involved in a construction project. Aconex is used by companies in various industries, including infrastructure, energy, mining, and real estate.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/c3eb7f52-9cb7-4501-8ae1-4651fba48937"
}
}
}, {
"id" : "9d66a65f-1e7b-4005-8056-8fcf2c4bbb2b",
"pluginType" : "EXTRACTOR",
"name" : "tap-activecampaign",
"namespace" : "tap_activecampaign",
"variant" : "singer-io",
"label" : "ActiveCampaign",
"description" : "ActiveCampaign is a customer experience automation platform that helps businesses connect and engage with their customers.\n\nActiveCampaign is a comprehensive platform that allows businesses to automate their marketing, sales, and customer service processes. It offers a range of features including email marketing, marketing automation, CRM, messaging, and more. With ActiveCampaign, businesses can create personalized experiences for their customers, automate repetitive tasks, and streamline their workflows. The platform also provides detailed analytics and insights to help businesses make data-driven decisions. Overall, ActiveCampaign helps businesses build stronger relationships with their customers and grow their business.",
"logoUrl" : "/assets/logos/extractors/activecampaign.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-activecampaign/",
"pipUrl" : "tap-activecampaign",
"repo" : "https://github.com/singer-io/tap-activecampaign",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "api_url",
"aliases" : [ ],
"label" : "API URL",
"options" : [ ],
"kind" : "STRING",
"description" : "The URL endpoint for the ActiveCampaign API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The date from which to retrieve data from ActiveCampaign.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "user_agent",
"aliases" : [ ],
"label" : "User Agent",
"options" : [ ],
"kind" : "STRING",
"description" : "A string identifying the client making the API request.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "api_token",
"aliases" : [ ],
"label" : "API Token",
"options" : [ ],
"kind" : "STRING",
"description" : "A unique identifier used to authenticate API requests to ActiveCampaign.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "ActiveCampaign is a customer experience automation platform that helps businesses connect and engage with their customers.\n\nActiveCampaign is a comprehensive platform that allows businesses to automate their marketing, sales, and customer service processes. It offers a range of features including email marketing, marketing automation, CRM, messaging, and more. With ActiveCampaign, businesses can create personalized experiences for their customers, automate repetitive tasks, and streamline their workflows. The platform also provides detailed analytics and insights to help businesses make data-driven decisions. Overall, ActiveCampaign helps businesses build stronger relationships with their customers and grow their business.\n\n## Settings\n\n\n### API URL\n\nThe URL endpoint for the ActiveCampaign API.\n\n### Start Date\n\nThe date from which to retrieve data from ActiveCampaign.\n\n### User Agent\n\nA string identifying the client making the API request.\n\n### API Token\n\nA unique identifier used to authenticate API requests to ActiveCampaign.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/9d66a65f-1e7b-4005-8056-8fcf2c4bbb2b"
}
}
}, {
"id" : "5e2f27a9-7cb3-4bfb-89ec-c4dad0bcb9bd",
"pluginType" : "EXTRACTOR",
"name" : "tap-acuite",
"namespace" : "tap_acuite",
"variant" : "fostersdata",
"label" : "Acuite",
"description" : "Acuite is a credit rating agency.\n\nAcuite provides credit ratings and risk assessments for companies and financial institutions in India, helping investors and lenders make informed decisions about creditworthiness and potential risks. Their ratings are based on a combination of quantitative and qualitative factors, including financial performance, industry trends, management quality, and governance practices. Acuite also offers research reports, industry analysis, and consulting services to support their clients' credit risk management needs.",
"logoUrl" : "/assets/logos/extractors/acuite.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-acuite/",
"pipUrl" : "git+https://github.com/FostersData/tap-acuite.git",
"repo" : "https://github.com/FostersData/tap-acuite",
"capabilities" : [ "DISCOVER", "PROPERTIES", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "api_key",
"aliases" : [ ],
"label" : "API Key",
"options" : [ ],
"kind" : "STRING",
"description" : "A unique identifier that authenticates and authorizes access to the Acuite API.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Acuite is a credit rating agency.\n\nAcuite provides credit ratings and risk assessments for companies and financial institutions in India, helping investors and lenders make informed decisions about creditworthiness and potential risks. Their ratings are based on a combination of quantitative and qualitative factors, including financial performance, industry trends, management quality, and governance practices. Acuite also offers research reports, industry analysis, and consulting services to support their clients' credit risk management needs.\n\n## Settings\n\n\n### API Key\n\nA unique identifier that authenticates and authorizes access to the Acuite API.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/5e2f27a9-7cb3-4bfb-89ec-c4dad0bcb9bd"
}
}
}, {
"id" : "0aff48c2-e6f1-473f-a0bf-a9e6ee5a8210",
"pluginType" : "EXTRACTOR",
"name" : "tap-adroll",
"namespace" : "tap_adroll",
"variant" : "singer-io",
"label" : "AdRoll",
"description" : "AdRoll is a digital marketing platform that helps businesses reach and engage with their target audience across various channels.\n\nAdRoll offers a suite of tools and services that enable businesses to create and launch targeted ad campaigns across multiple channels, including social media, email, and display advertising. The platform uses machine learning algorithms to optimize ad performance and improve ROI, while also providing detailed analytics and reporting to help businesses track their results and make data-driven decisions. AdRoll also offers a range of other services, including retargeting, email marketing, and audience segmentation, to help businesses reach and engage with their target audience more effectively.",
"logoUrl" : "/assets/logos/extractors/adroll.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-adroll/",
"pipUrl" : "git+https://github.com/singer-io/tap-adroll.git",
"repo" : "https://github.com/singer-io/tap-adroll",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The date from which data will be retrieved.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "access_token",
"aliases" : [ ],
"label" : "Access Token",
"options" : [ ],
"kind" : "STRING",
"description" : "A token that grants access to the AdRoll API.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "refresh_token",
"aliases" : [ ],
"label" : "Refresh Token",
"options" : [ ],
"kind" : "STRING",
"description" : "A token used to refresh the access token when it expires.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "client_id",
"aliases" : [ ],
"label" : "Client ID",
"options" : [ ],
"kind" : "STRING",
"description" : "A unique identifier for the client application.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "client_secret",
"aliases" : [ ],
"label" : "Client Secret",
"options" : [ ],
"kind" : "STRING",
"description" : "A secret key used to authenticate the client application.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "lookback_window",
"aliases" : [ ],
"label" : "Lookback Window",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The time period in which AdRoll will look back to attribute conversions to a particular ad.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "end_date",
"aliases" : [ ],
"label" : "End Date",
"options" : [ ],
"kind" : "STRING",
"description" : "The date up to which data will be retrieved.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "AdRoll is a digital marketing platform that helps businesses reach and engage with their target audience across various channels.\n\nAdRoll offers a suite of tools and services that enable businesses to create and launch targeted ad campaigns across multiple channels, including social media, email, and display advertising. The platform uses machine learning algorithms to optimize ad performance and improve ROI, while also providing detailed analytics and reporting to help businesses track their results and make data-driven decisions. AdRoll also offers a range of other services, including retargeting, email marketing, and audience segmentation, to help businesses reach and engage with their target audience more effectively.\n\n## Settings\n\n\n### Start Date\n\nThe date from which data will be retrieved.\n\n### Access Token\n\nA token that grants access to the AdRoll API.\n\n### Refresh Token\n\nA token used to refresh the access token when it expires.\n\n### Client ID\n\nA unique identifier for the client application.\n\n### Client Secret\n\nA secret key used to authenticate the client application.\n\n### Lookback Window\n\nThe time period in which AdRoll will look back to attribute conversions to a particular ad.\n\n### End Date\n\nThe date up to which data will be retrieved.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/0aff48c2-e6f1-473f-a0bf-a9e6ee5a8210"
}
}
}, {
"id" : "bc600a33-49a6-4035-8900-e6f783903857",
"pluginType" : "EXTRACTOR",
"name" : "tap-adyen",
"namespace" : "tap_adyen",
"variant" : "yoast",
"label" : "Adyen",
"description" : "Adyen is a payment platform that enables businesses to accept and process payments from customers across various channels and geographies. \n\nAdyen provides a unified payment solution that allows businesses to accept payments from customers across various channels, including online, mobile, and in-store. The platform supports over 250 payment methods and 150 currencies, making it easy for businesses to expand their reach globally. Adyen also offers a range of features such as fraud detection, recurring payments, and real-time reporting to help businesses manage their payments efficiently. With Adyen, businesses can provide a seamless payment experience to their customers while reducing the complexity and costs associated with managing multiple payment providers.",
"logoUrl" : "/assets/logos/extractors/adyen.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-adyen/",
"pipUrl" : "git+https://github.com/Yoast/singer-tap-adyen.git",
"repo" : "https://github.com/Yoast/singer-tap-adyen",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "company_account",
"aliases" : [ ],
"label" : "Company Account",
"options" : [ ],
"kind" : "STRING",
"description" : "The name of the company's Adyen account.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The date from which to start generating reports.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "report_user",
"aliases" : [ ],
"label" : "Report User",
"options" : [ ],
"kind" : "STRING",
"description" : "The username of the user who will generate the report.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "user_password",
"aliases" : [ ],
"label" : "User Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password for the report user.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "merchant_account",
"aliases" : [ ],
"label" : "Merchant Account",
"options" : [ ],
"kind" : "STRING",
"description" : "The specific merchant account to use for the transaction.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "test",
"aliases" : [ ],
"label" : "Test",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "A boolean value indicating whether to use the Adyen test environment or the live environment.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Adyen is a payment platform that enables businesses to accept and process payments from customers across various channels and geographies. \n\nAdyen provides a unified payment solution that allows businesses to accept payments from customers across various channels, including online, mobile, and in-store. The platform supports over 250 payment methods and 150 currencies, making it easy for businesses to expand their reach globally. Adyen also offers a range of features such as fraud detection, recurring payments, and real-time reporting to help businesses manage their payments efficiently. With Adyen, businesses can provide a seamless payment experience to their customers while reducing the complexity and costs associated with managing multiple payment providers.\n\n## Settings\n\n\n### Company Account\n\nThe name of the company's Adyen account.\n\n### Start Date\n\nThe date from which to start generating reports.\n\n### Report User\n\nThe username of the user who will generate the report.\n\n### User Password\n\nThe password for the report user.\n\n### Merchant Account\n\nThe specific merchant account to use for the transaction.\n\n### Test\n\nA boolean value indicating whether to use the Adyen test environment or the live environment.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/bc600a33-49a6-4035-8900-e6f783903857"
}
}
}, {
"id" : "7bb956c3-9f8f-4f5e-94a0-936c6a5376f0",
"pluginType" : "EXTRACTOR",
"name" : "tap-affinity",
"namespace" : "tap_affinity",
"variant" : "hotgluexyz",
"label" : "Affinity",
"description" : "Affinity is a suite of creative software tools.\n\nAffinity offers a range of software tools for creative professionals, including photo editing, graphic design, and desktop publishing. Their software is designed to be affordable and accessible, while still offering powerful features and professional-grade results. Affinity's tools are available for both Mac and Windows platforms, and they offer a range of tutorials and resources to help users get the most out of their software.",
"logoUrl" : "/assets/logos/extractors/affinity.svg",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-affinity/",
"pipUrl" : "git+https://gitlab.com/hotglue/tap-affinity.git",
"repo" : "https://gitlab.com/hotglue/tap-affinity",
"executable" : "tap-affinity",
"capabilities" : [ "CATALOG", "STREAM_MAPS", "DISCOVER", "STATE", "ABOUT", "SCHEMA_FLATTENING" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "api_key",
"aliases" : [ ],
"label" : "API Key",
"options" : [ ],
"kind" : "STRING",
"description" : "A unique identifier used to authenticate and authorize access to the Affinity API.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_enabled",
"aliases" : [ ],
"label" : "Flattening Enabled",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "A boolean value indicating whether or not to flatten nested objects in the API response.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_max_depth",
"aliases" : [ ],
"label" : "Flattening Max Depth",
"options" : [ ],
"kind" : "INTEGER",
"description" : "An integer value indicating the maximum depth of nested objects to flatten in the API response.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_map_config",
"aliases" : [ ],
"label" : "Stream Map Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "A configuration file that maps the API response to a desired output format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_maps",
"aliases" : [ ],
"label" : "Stream Maps",
"options" : [ ],
"kind" : "OBJECT",
"description" : "A collection of mappings between API response fields and output fields defined in the Stream Map Config.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Affinity is a suite of creative software tools.\n\nAffinity offers a range of software tools for creative professionals, including photo editing, graphic design, and desktop publishing. Their software is designed to be affordable and accessible, while still offering powerful features and professional-grade results. Affinity's tools are available for both Mac and Windows platforms, and they offer a range of tutorials and resources to help users get the most out of their software.\n\n## Settings\n\n\n### API Key\n\nA unique identifier used to authenticate and authorize access to the Affinity API.\n\n### Flattening Enabled\n\nA boolean value indicating whether or not to flatten nested objects in the API response.\n\n### Flattening Max Depth\n\nAn integer value indicating the maximum depth of nested objects to flatten in the API response.\n\n### Stream Map Config\n\nA configuration file that maps the API response to a desired output format.\n\n### Stream Maps\n\nA collection of mappings between API response fields and output fields defined in the Stream Map Config.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/7bb956c3-9f8f-4f5e-94a0-936c6a5376f0"
}
}
}, {
"id" : "a163cb3e-85ea-463b-aec6-d822fc059b5c",
"pluginType" : "EXTRACTOR",
"name" : "tap-aftership",
"namespace" : "tap_aftership",
"variant" : "harrystech",
"label" : "AfterShip",
"description" : "AfterShip is a shipment tracking platform for online retailers and customers.\n\nAfterShip allows online retailers to track and manage their shipments across multiple carriers and provides customers with real-time updates on the status of their orders. The platform integrates with over 700 carriers worldwide and offers features such as branded tracking pages, delivery notifications, and analytics to help businesses improve their shipping performance. AfterShip also offers a mobile app for customers to track their packages on-the-go.",
"logoUrl" : "/assets/logos/extractors/aftership.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-aftership/",
"pipUrl" : "git+https://github.com/harrystech/tap-aftership.git",
"repo" : "https://github.com/harrystech/tap-aftership",
"capabilities" : [ "CATALOG", "STREAM_MAPS", "DISCOVER", "STATE", "ABOUT", "SCHEMA_FLATTENING" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "api_key",
"aliases" : [ ],
"label" : "Api Key",
"options" : [ ],
"kind" : "STRING",
"description" : "A unique identifier used to authenticate and authorize API requests.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The earliest date for which shipment tracking information should be retrieved.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "end_date",
"aliases" : [ ],
"label" : "End Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The latest date for which shipment tracking information should be retrieved.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_maps",
"aliases" : [ ],
"label" : "Stream Maps",
"options" : [ ],
"kind" : "OBJECT",
"description" : "A list of stream maps that define the structure of the response data.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_map_config",
"aliases" : [ ],
"label" : "Stream Map Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "Additional configuration settings for the stream maps.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_enabled",
"aliases" : [ ],
"label" : "Flattening Enabled",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "A boolean value indicating whether or not the response data should be flattened.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_max_depth",
"aliases" : [ ],
"label" : "Flattening Max Depth",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum depth to which the response data should be flattened.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "AfterShip is a shipment tracking platform for online retailers and customers.\n\nAfterShip allows online retailers to track and manage their shipments across multiple carriers and provides customers with real-time updates on the status of their orders. The platform integrates with over 700 carriers worldwide and offers features such as branded tracking pages, delivery notifications, and analytics to help businesses improve their shipping performance. AfterShip also offers a mobile app for customers to track their packages on-the-go.\n\n## Settings\n\n\n### Api Key\n\nA unique identifier used to authenticate and authorize API requests.\n\n### Start Date\n\nThe earliest date for which shipment tracking information should be retrieved.\n\n### End Date\n\nThe latest date for which shipment tracking information should be retrieved.\n\n### Stream Maps\n\nA list of stream maps that define the structure of the response data.\n\n### Stream Map Config\n\nAdditional configuration settings for the stream maps.\n\n### Flattening Enabled\n\nA boolean value indicating whether or not the response data should be flattened.\n\n### Flattening Max Depth\n\nThe maximum depth to which the response data should be flattened.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/a163cb3e-85ea-463b-aec6-d822fc059b5c"
}
}
}, {
"id" : "d5999190-7b6b-4c01-8a39-e6da47dfc88f",
"pluginType" : "EXTRACTOR",
"name" : "tap-agilecrm",
"namespace" : "tap_agilecrm",
"variant" : "dreamdata-io",
"label" : "Agile CRM",
"description" : "Agile CRM is a customer relationship management software that helps businesses manage their sales, marketing, and customer service activities in one platform.\n\nAgile CRM is designed to streamline customer interactions by providing a centralized platform for managing sales, marketing, and customer service activities. It offers features such as contact management, lead scoring, email marketing, social media integration, and analytics to help businesses improve their customer engagement and increase sales. The software also includes automation tools to help businesses automate repetitive tasks and workflows, freeing up time for more important tasks. Additionally, Agile CRM offers integrations with popular third-party tools such as Zapier, Slack, and Shopify, making it a versatile solution for businesses of all sizes.",
"logoUrl" : "/assets/logos/extractors/agilecrm.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-agilecrm/",
"pipUrl" : "git+https://github.com/dreamdata-io/tap-agilecrm.git",
"repo" : "https://github.com/dreamdata-io/tap-agilecrm",
"capabilities" : [ "CATALOG", "DISCOVER" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Agile CRM is a customer relationship management software that helps businesses manage their sales, marketing, and customer service activities in one platform.\n\nAgile CRM is designed to streamline customer interactions by providing a centralized platform for managing sales, marketing, and customer service activities. It offers features such as contact management, lead scoring, email marketing, social media integration, and analytics to help businesses improve their customer engagement and increase sales. The software also includes automation tools to help businesses automate repetitive tasks and workflows, freeing up time for more important tasks. Additionally, Agile CRM offers integrations with popular third-party tools such as Zapier, Slack, and Shopify, making it a versatile solution for businesses of all sizes.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/d5999190-7b6b-4c01-8a39-e6da47dfc88f"
}
}
}, {
"id" : "06c5cf55-b9fb-4a50-9ce9-bbc8221f0318",
"pluginType" : "EXTRACTOR",
"name" : "tap-airbyte-wrapper",
"namespace" : "tap_airbyte_wrapper",
"variant" : "meltanolabs",
"label" : "Airbyte Protocol Wrapper",
"description" : "Airbyte Protocol Wrapper is a tool that enables data integration between different systems and applications.\n\nAirbyte Protocol Wrapper is an open-source tool that provides a standardized way of integrating data between different systems and applications. It acts as a bridge between different data sources and destinations, allowing users to easily move data between them. The tool supports a wide range of data sources and destinations, including databases, APIs, and file systems. It also provides a simple and intuitive interface for configuring data integration workflows, making it easy for users to set up and manage their data pipelines. With Airbyte Protocol Wrapper, users can streamline their data integration processes and ensure that their data is always up-to-date and accurate.",
"logoUrl" : "/assets/logos/extractors/airbyte.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-airbyte-wrapper/",
"pipUrl" : "git+https://github.com/meltanolabs/tap-airbyte-wrapper.git",
"repo" : "https://github.com/meltanolabs/tap-airbyte-wrapper",
"executable" : "tap-airbyte-wrapper",
"capabilities" : [ "CATALOG", "STREAM_MAPS", "DISCOVER", "STATE", "ABOUT", "SCHEMA_FLATTENING" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "airbyte_spec.image",
"aliases" : [ ],
"label" : "Airbyte Spec Image",
"options" : [ ],
"kind" : "STRING",
"description" : "The Docker image containing the Airbyte specification for the connector.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "airbyte_spec.tag",
"aliases" : [ ],
"label" : "Airbyte Spec Tag",
"options" : [ ],
"kind" : "STRING",
"description" : "The specific version of the Airbyte specification to use.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "airbyte_config",
"aliases" : [ ],
"label" : "Airbyte Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "The configuration settings for the connector, such as API keys and credentials.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "docker_mounts",
"aliases" : [ ],
"label" : "Docker Mounts",
"options" : [ ],
"kind" : "ARRAY",
"description" : "The file system paths to mount as volumes in the Docker container.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_maps",
"aliases" : [ ],
"label" : "Stream Maps",
"options" : [ ],
"kind" : "OBJECT",
"description" : "The mapping of source data streams to destination data streams.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_map_config",
"aliases" : [ ],
"label" : "Stream Map Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "Additional configuration settings for the stream maps.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_enabled",
"aliases" : [ ],
"label" : "Flattening Enabled",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to flatten nested data structures.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_max_depth",
"aliases" : [ ],
"label" : "Flattening Max Depth",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum depth to which nested data structures will be flattened.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Airbyte Protocol Wrapper is a tool that enables data integration between different systems and applications.\n\nAirbyte Protocol Wrapper is an open-source tool that provides a standardized way of integrating data between different systems and applications. It acts as a bridge between different data sources and destinations, allowing users to easily move data between them. The tool supports a wide range of data sources and destinations, including databases, APIs, and file systems. It also provides a simple and intuitive interface for configuring data integration workflows, making it easy for users to set up and manage their data pipelines. With Airbyte Protocol Wrapper, users can streamline their data integration processes and ensure that their data is always up-to-date and accurate.\n\n## Settings\n\n\n### Airbyte Spec Image\n\nThe Docker image containing the Airbyte specification for the connector.\n\n### Airbyte Spec Tag\n\nThe specific version of the Airbyte specification to use.\n\n### Airbyte Config\n\nThe configuration settings for the connector, such as API keys and credentials.\n\n### Docker Mounts\n\nThe file system paths to mount as volumes in the Docker container.\n\n### Stream Maps\n\nThe mapping of source data streams to destination data streams.\n\n### Stream Map Config\n\nAdditional configuration settings for the stream maps.\n\n### Flattening Enabled\n\nWhether or not to flatten nested data structures.\n\n### Flattening Max Depth\n\nThe maximum depth to which nested data structures will be flattened.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/06c5cf55-b9fb-4a50-9ce9-bbc8221f0318"
}
}
}, {
"id" : "87e47a5c-175e-465b-b8cb-72f6e8cef137",
"pluginType" : "EXTRACTOR",
"name" : "tap-beautifulsoup",
"namespace" : "tap_beautifulsoup",
"variant" : "matatika",
"label" : "BeautifulSoup",
"description" : "Python library for pulling data out of HTML and XML files.",
"logoUrl" : "/assets/images/datasource/tap-beautifulsoup.png",
"hidden" : false,
"pipUrl" : "git+https://github.com/Matatika/tap-beautifulsoup.git",
"repo" : "https://github.com/Matatika/tap-beautifulsoup",
"executable" : "tap-beautifulsoup",
"capabilities" : [ "CATALOG", "STREAM_MAPS", "DISCOVER", "STATE", "ABOUT", "SCHEMA_FLATTENING" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "download_recursively",
"aliases" : [ ],
"label" : "Download Recursively",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Attempt to download all pages recursively into the output directory prior to parsing files. Set this to False if you've previously run `wget -r -A. Html https://sdk.meltano.com/en/latest/`",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "exclude_tags",
"aliases" : [ ],
"label" : "Exclude tags",
"value" : "[\"header\",\"title\",\"aside\",\"label\",\"button\",\"footer\",\"style\"]",
"options" : [ ],
"kind" : "ARRAY",
"description" : "List of tags to exclude before extracting text content of the page.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "find_all_kwargs",
"aliases" : [ ],
"label" : "Find All Kwargs",
"value" : "{\"name\": \"body\"}",
"options" : [ ],
"kind" : "OBJECT",
"description" : "This dict contains all the kwargs that should be passed to the [`find_all`](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#find-all) call in order to extract text from the pages.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_enabled",
"aliases" : [ ],
"label" : "Flattening Enabled",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "'True' to enable schema flattening and automatically expand nested properties.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_max_depth",
"aliases" : [ ],
"label" : "Flattening Max Depth",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The max depth to flatten schemas.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "output_folder",
"aliases" : [ ],
"label" : "Output Folder",
"value" : "output",
"options" : [ ],
"kind" : "STRING",
"description" : "The file path of where to write the intermediate downloaded HTML files to.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "parser",
"aliases" : [ ],
"label" : "Parser",
"value" : "html.parser",
"options" : [ {
"label" : "Html Parser",
"value" : "html.parser"
} ],
"kind" : "OPTIONS",
"description" : "The BeautifulSoup parser to use.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "site_url",
"aliases" : [ ],
"label" : "Site URL",
"options" : [ ],
"kind" : "STRING",
"description" : "The site you'd like to scrape. The tap will download all pages recursively into the output directory prior to parsing files.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "source_name",
"aliases" : [ ],
"label" : "Source Name",
"options" : [ ],
"kind" : "STRING",
"description" : "The name of the source you're scraping. This will be used as the stream name.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_map_config",
"aliases" : [ ],
"label" : "Stream Map Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "User-defined config values to be used within map expressions.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_maps",
"aliases" : [ ],
"label" : "Stream Maps",
"options" : [ ],
"kind" : "OBJECT",
"description" : "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html).",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Python library for pulling data out of HTML and XML files.\n\n## Settings\n\n\n### Download Recursively\n\nAttempt to download all pages recursively into the output directory prior to parsing files. Set this to False if you've previously run `wget -r -A. Html https://sdk.meltano.com/en/latest/`\n\n### Exclude tags\n\nList of tags to exclude before extracting text content of the page.\n\n### Find All Kwargs\n\nThis dict contains all the kwargs that should be passed to the [`find_all`](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#find-all) call in order to extract text from the pages.\n\n### Flattening Enabled\n\n'True' to enable schema flattening and automatically expand nested properties.\n\n### Flattening Max Depth\n\nThe max depth to flatten schemas.\n\n### Output Folder\n\nThe file path of where to write the intermediate downloaded HTML files to.\n\n### Parser\n\nThe BeautifulSoup parser to use.\n\n### Site URL\n\nThe site you'd like to scrape. The tap will download all pages recursively into the output directory prior to parsing files.\n\n### Source Name\n\nThe name of the source you're scraping. This will be used as the stream name.\n\n### Stream Map Config\n\nUser-defined config values to be used within map expressions.\n\n### Stream Maps\n\nConfig object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html).",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/87e47a5c-175e-465b-b8cb-72f6e8cef137"
}
}
}, {
"id" : "0732e846-10ac-433b-bef0-546301304db2",
"pluginType" : "EXTRACTOR",
"name" : "tap-airtable",
"namespace" : "tap_airtable",
"variant" : "goes-funky",
"label" : "Airtable",
"description" : "Airtable is a cloud-based software that combines the functionality of a spreadsheet with the power of a database, allowing users to organize and manage data in a customizable and collaborative way.\n\nAirtable provides a user-friendly interface for creating and managing databases, with features such as drag-and-drop fields, customizable views, and real-time collaboration. Users can create tables to store data, link records between tables, and filter, sort, and group data in various ways. Airtable also offers integrations with other tools such as Slack, Zapier, and Google Drive, making it a versatile tool for managing projects, tracking inventory, organizing customer data, and more. With its flexibility and ease of use, Airtable is a popular choice for teams and individuals looking to streamline their data management processes.",
"logoUrl" : "/assets/logos/extractors/airtable.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-airtable/",
"pipUrl" : "git+https://github.com/goes-funky/tap-airtable.git",
"repo" : "https://github.com/goes-funky/tap-airtable",
"capabilities" : [ "DISCOVER", "PROPERTIES" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "token",
"aliases" : [ ],
"label" : "Token",
"options" : [ ],
"kind" : "STRING",
"description" : "An authentication key used to access the Airtable API.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "metadata_url",
"aliases" : [ ],
"label" : "Metadata URL",
"options" : [ ],
"kind" : "STRING",
"description" : "The URL used to retrieve metadata about a specific Airtable base.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "records_url",
"aliases" : [ ],
"label" : "Records URL",
"options" : [ ],
"kind" : "STRING",
"description" : "The URL used to retrieve records from a specific Airtable base.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "base_id",
"aliases" : [ ],
"label" : "Base ID",
"options" : [ ],
"kind" : "STRING",
"description" : "A unique identifier for a specific Airtable base.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "selected_by_default",
"aliases" : [ ],
"label" : "Selected By Default",
"options" : [ ],
"kind" : "STRING",
"description" : "A boolean value indicating whether a specific option is selected by default.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "remove_emojis",
"aliases" : [ ],
"label" : "Remove Emojis",
"options" : [ ],
"kind" : "STRING",
"description" : "A boolean value indicating whether emojis should be removed from the data retrieved from Airtable.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Airtable is a cloud-based software that combines the functionality of a spreadsheet with the power of a database, allowing users to organize and manage data in a customizable and collaborative way.\n\nAirtable provides a user-friendly interface for creating and managing databases, with features such as drag-and-drop fields, customizable views, and real-time collaboration. Users can create tables to store data, link records between tables, and filter, sort, and group data in various ways. Airtable also offers integrations with other tools such as Slack, Zapier, and Google Drive, making it a versatile tool for managing projects, tracking inventory, organizing customer data, and more. With its flexibility and ease of use, Airtable is a popular choice for teams and individuals looking to streamline their data management processes.\n\n## Settings\n\n\n### Token\n\nAn authentication key used to access the Airtable API.\n\n### Metadata URL\n\nThe URL used to retrieve metadata about a specific Airtable base.\n\n### Records URL\n\nThe URL used to retrieve records from a specific Airtable base.\n\n### Base ID\n\nA unique identifier for a specific Airtable base.\n\n### Selected By Default\n\nA boolean value indicating whether a specific option is selected by default.\n\n### Remove Emojis\n\nA boolean value indicating whether emojis should be removed from the data retrieved from Airtable.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/0732e846-10ac-433b-bef0-546301304db2"
}
}
}, {
"id" : "246cb678-a8ad-4311-ab37-bcb8776dd103",
"pluginType" : "EXTRACTOR",
"name" : "tap-amazon-ads-dsp",
"namespace" : "tap_amazon_ads_dsp",
"variant" : "singer-io",
"label" : "Amazon Advertising DSP",
"description" : "Amazon Advertising DSP is a demand-side platform that allows advertisers to programmatically buy display and video ads across Amazon's inventory and other third-party exchanges.\n\nAmazon Advertising DSP provides advertisers with a self-service platform to programmatically buy display and video ads across Amazon's inventory and other third-party exchanges. The platform offers advanced targeting options, including audience targeting, contextual targeting, and retargeting, as well as real-time reporting and optimization tools to help advertisers maximize their ad performance. With Amazon Advertising DSP, advertisers can reach their target audience at scale and drive brand awareness, consideration, and sales.",
"logoUrl" : "/assets/logos/extractors/amazon-ads-dsp.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-amazon-ads-dsp/",
"pipUrl" : "tap-amazon-ads-dsp",
"repo" : "https://github.com/singer-io/tap-amazon-ads-dsp",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Amazon Advertising DSP is a demand-side platform that allows advertisers to programmatically buy display and video ads across Amazon's inventory and other third-party exchanges.\n\nAmazon Advertising DSP provides advertisers with a self-service platform to programmatically buy display and video ads across Amazon's inventory and other third-party exchanges. The platform offers advanced targeting options, including audience targeting, contextual targeting, and retargeting, as well as real-time reporting and optimization tools to help advertisers maximize their ad performance. With Amazon Advertising DSP, advertisers can reach their target audience at scale and drive brand awareness, consideration, and sales.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/246cb678-a8ad-4311-ab37-bcb8776dd103"
}
}
}, {
"id" : "21072f8c-c47a-4d88-826d-b5021f55509c",
"pluginType" : "EXTRACTOR",
"name" : "tap-amazon-advertising",
"namespace" : "tap_amazon_advertising",
"variant" : "dbt-labs",
"label" : "Amazon Advertising",
"description" : "Amazon Advertising is a platform that allows businesses to advertise their products and services on Amazon's website and other affiliated sites.\n\nAmazon Advertising provides businesses with a variety of advertising options to help them reach their target audience on Amazon's website and other affiliated sites. These options include sponsored products, sponsored brands, and sponsored display ads. With Amazon Advertising, businesses can create and manage their advertising campaigns, set their own budgets, and track their performance using detailed analytics. This platform also provides businesses with access to Amazon's vast customer base, allowing them to reach millions of potential customers around the world.",
"logoUrl" : "/assets/logos/extractors/amazon-advertising.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-amazon-advertising/",
"pipUrl" : "git+https://github.com/dbt-labs/tap-amazon-advertising.git",
"repo" : "https://github.com/dbt-labs/tap-amazon-advertising",
"capabilities" : [ "CATALOG", "DISCOVER", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "profile_id",
"aliases" : [ ],
"label" : "Profile ID",
"options" : [ ],
"kind" : "STRING",
"description" : "The unique identifier for the Amazon Advertising profile you want to access.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "DATE_ISO8601",
"description" : "The date from which you want to retrieve data.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "client_secret",
"aliases" : [ ],
"label" : "Client Secret",
"options" : [ ],
"kind" : "STRING",
"description" : "A secret key used to authenticate your application when making API requests.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "client_id",
"aliases" : [ ],
"label" : "Client ID",
"options" : [ ],
"kind" : "STRING",
"description" : "A unique identifier for your application when making API requests.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "refresh_token",
"aliases" : [ ],
"label" : "Refresh Token",
"options" : [ ],
"kind" : "STRING",
"description" : "A token used to obtain a new access token when the original token expires.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "redirect_uri",
"aliases" : [ ],
"label" : "Redirect URI",
"options" : [ ],
"kind" : "STRING",
"description" : "The URI to which the user will be redirected after granting permission to your application.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Amazon Advertising is a platform that allows businesses to advertise their products and services on Amazon's website and other affiliated sites.\n\nAmazon Advertising provides businesses with a variety of advertising options to help them reach their target audience on Amazon's website and other affiliated sites. These options include sponsored products, sponsored brands, and sponsored display ads. With Amazon Advertising, businesses can create and manage their advertising campaigns, set their own budgets, and track their performance using detailed analytics. This platform also provides businesses with access to Amazon's vast customer base, allowing them to reach millions of potential customers around the world.\n\n## Settings\n\n\n### Profile ID\n\nThe unique identifier for the Amazon Advertising profile you want to access.\n\n### Start Date\n\nThe date from which you want to retrieve data.\n\n### Client Secret\n\nA secret key used to authenticate your application when making API requests.\n\n### Client ID\n\nA unique identifier for your application when making API requests.\n\n### Refresh Token\n\nA token used to obtain a new access token when the original token expires.\n\n### Redirect URI\n\nThe URI to which the user will be redirected after granting permission to your application.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/21072f8c-c47a-4d88-826d-b5021f55509c"
}
}
}, {
"id" : "48522ca7-3c0c-4b56-a5d8-af2ef443ca41",
"pluginType" : "EXTRACTOR",
"name" : "tap-amazon-associates",
"namespace" : "tap_amazon_associates",
"variant" : "voxmedia",
"label" : "Amazon Associates",
"description" : "Amazon Associates is a program that allows website owners and bloggers to earn commissions by promoting Amazon products on their sites.\n\nAmazon Associates is an affiliate marketing program that enables website owners and bloggers to earn money by promoting Amazon products on their sites. Participants can earn up to 10% in referral fees for qualifying purchases made through their unique affiliate links. The program provides a variety of tools and resources to help affiliates promote products, including customizable banners and widgets, product links, and a product advertising API. Amazon Associates also offers reporting and analytics tools to help affiliates track their earnings and optimize their campaigns.",
"logoUrl" : "/assets/logos/extractors/amazon-associates.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-amazon-associates/",
"pipUrl" : "git+https://github.com/voxmedia/tap-amazon-associates.git",
"repo" : "https://github.com/voxmedia/tap-amazon-associates",
"capabilities" : [ "CATALOG", "STREAM_MAPS", "DISCOVER", "STATE", "ABOUT", "SCHEMA_FLATTENING" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "username",
"aliases" : [ ],
"label" : "Username",
"options" : [ ],
"kind" : "STRING",
"description" : "The username used to authenticate with the Amazon Associates API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "password",
"aliases" : [ ],
"label" : "Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password used to authenticate with the Amazon Associates API.",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"kind" : "STRING",
"description" : "The date from which to start retrieving data from the Amazon Associates API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "api_url",
"aliases" : [ ],
"label" : "Api Url",
"options" : [ ],
"kind" : "STRING",
"description" : "The URL of the Amazon Associates API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_maps",
"aliases" : [ ],
"label" : "Stream Maps",
"options" : [ ],
"kind" : "OBJECT",
"description" : "A list of stream maps to use when retrieving data from the Amazon Associates API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "stream_map_config",
"aliases" : [ ],
"label" : "Stream Map Config",
"options" : [ ],
"kind" : "OBJECT",
"description" : "Configuration settings for the stream maps used when retrieving data from the Amazon Associates API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_enabled",
"aliases" : [ ],
"label" : "Flattening Enabled",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to flatten the data retrieved from the Amazon Associates API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flattening_max_depth",
"aliases" : [ ],
"label" : "Flattening Max Depth",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum depth to which data should be flattened when retrieving data from the Amazon Associates API.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Amazon Associates is a program that allows website owners and bloggers to earn commissions by promoting Amazon products on their sites.\n\nAmazon Associates is an affiliate marketing program that enables website owners and bloggers to earn money by promoting Amazon products on their sites. Participants can earn up to 10% in referral fees for qualifying purchases made through their unique affiliate links. The program provides a variety of tools and resources to help affiliates promote products, including customizable banners and widgets, product links, and a product advertising API. Amazon Associates also offers reporting and analytics tools to help affiliates track their earnings and optimize their campaigns.\n\n## Settings\n\n\n### Username\n\nThe username used to authenticate with the Amazon Associates API.\n\n### Password\n\nThe password used to authenticate with the Amazon Associates API.\n\n### Start Date\n\nThe date from which to start retrieving data from the Amazon Associates API.\n\n### Api Url\n\nThe URL of the Amazon Associates API.\n\n### Stream Maps\n\nA list of stream maps to use when retrieving data from the Amazon Associates API.\n\n### Stream Map Config\n\nConfiguration settings for the stream maps used when retrieving data from the Amazon Associates API.\n\n### Flattening Enabled\n\nWhether or not to flatten the data retrieved from the Amazon Associates API.\n\n### Flattening Max Depth\n\nThe maximum depth to which data should be flattened when retrieving data from the Amazon Associates API.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/48522ca7-3c0c-4b56-a5d8-af2ef443ca41"
}
}
}, {
"id" : "c1bf5c8d-06dd-436d-86fa-7ba5dd4b1f29",
"pluginType" : "EXTRACTOR",
"name" : "tap-amazon-mws",
"namespace" : "tap_amazon_mws",
"variant" : "dbt-labs",
"label" : "Amazon MWS",
"description" : "Amazon MWS is a suite of APIs that allows sellers to automate their selling workflows on Amazon.\n\nAmazon Marketplace Web Service (MWS) is a platform that sellers can use to automate their selling workflows on Amazon. MWS provides APIs that allow sellers to programmatically exchange data on listings, orders, payments, reports, and more. With MWS, sellers can integrate their Amazon selling operations with their own systems, enabling them to manage their inventory, fulfill orders, and track sales more efficiently. MWS also provides tools for managing pricing, promotions, and advertising campaigns, as well as for monitoring performance metrics and customer feedback. Overall, MWS helps sellers streamline their Amazon selling operations and grow their businesses.",
"logoUrl" : "/assets/logos/extractors/amazon-mws.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/tap-amazon-mws/",
"pipUrl" : "git+https://github.com/dbt-labs/tap-amazon-mws.git",
"repo" : "https://github.com/dbt-labs/tap-amazon-mws",
"capabilities" : [ "CATALOG", "DISCOVER" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Amazon MWS is a suite of APIs that allows sellers to automate their selling workflows on Amazon.\n\nAmazon Marketplace Web Service (MWS) is a platform that sellers can use to automate their selling workflows on Amazon. MWS provides APIs that allow sellers to programmatically exchange data on listings, orders, payments, reports, and more. With MWS, sellers can integrate their Amazon selling operations with their own systems, enabling them to manage their inventory, fulfill orders, and track sales more efficiently. MWS also provides tools for managing pricing, promotions, and advertising campaigns, as well as for monitoring performance metrics and customer feedback. Overall, MWS helps sellers streamline their Amazon selling operations and grow their businesses.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/c1bf5c8d-06dd-436d-86fa-7ba5dd4b1f29"
}
}
} ]
},
"_links" : {
"first" : {
"href" : "https://staging.meltano.com/api/dataplugins?page=0&size=20"
},
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins?page=0&size=20"
},
"next" : {
"href" : "https://staging.meltano.com/api/dataplugins?page=1&size=20"
},
"last" : {
"href" : "https://staging.meltano.com/api/dataplugins?page=27&size=20"
}
},
"page" : {
"size" : 20,
"totalElements" : 554,
"totalPages" : 28,
"number" : 0
}
}
View all workspace dataplugins
GET /api/workspaces/{workspace-id}/dataplugins
Returns all dataplugins available to the workspace {workspace-id}.
Prerequisites
- Workspace
{workspace-id}must exist
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins' -i -X GET
import requests
url = "https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Dataplugin collection with HAL links.
{
"_embedded" : {
"dataplugins" : [ {
"id" : "32374218-07a4-4a49-92f6-476339ae7b31",
"pluginType" : "LOADER",
"name" : "target-postgres",
"namespace" : "postgres_transferwise",
"variant" : "matatika",
"label" : "Postgres Warehouse",
"description" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.",
"logoUrl" : "/assets/logos/loaders/postgres.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/target-postgres/",
"pipUrl" : "git+https://github.com/Matatika/pipelinewise-target-postgres.git@v0.2.1",
"repo" : "https://github.com/Matatika/pipelinewise-target-postgres",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "user",
"aliases" : [ "username" ],
"label" : "User",
"options" : [ ],
"kind" : "STRING",
"description" : "The username used to connect to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "password",
"aliases" : [ ],
"label" : "Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password used to authenticate the user.",
"hidden" : false,
"sensitive" : true,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "host",
"aliases" : [ "address" ],
"label" : "Host",
"options" : [ ],
"kind" : "STRING",
"description" : "The hostname or IP address of the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "port",
"aliases" : [ ],
"label" : "Port",
"value" : "5432",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The port number used to connect to the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "dbname",
"aliases" : [ "database" ],
"label" : "Database Name",
"options" : [ ],
"kind" : "STRING",
"description" : "The name of the database to connect to.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "default_target_schema",
"aliases" : [ ],
"label" : "Default Target Schema",
"value" : "analytics",
"options" : [ ],
"kind" : "STRING",
"description" : "The default schema to use when writing data to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "state_schema",
"aliases" : [ ],
"label" : "State Schema",
"value" : "public",
"options" : [ ],
"kind" : "STRING",
"description" : "The default schema to use when writing Meltano state.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "ssl",
"aliases" : [ ],
"label" : "SSL",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to use SSL encryption when connecting to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false,
"value_post_processor" : "STRINGIFY"
}, {
"name" : "batch_size_rows",
"aliases" : [ ],
"label" : "Batch Size Rows",
"value" : "100000",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of rows to write to the Postgres Warehouse in each batch.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "underscore_camel_case_fields",
"aliases" : [ ],
"label" : "Underscore Camel Case Fields",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to convert field names from camel case to underscore-separated format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flush_all_streams",
"aliases" : [ ],
"label" : "Flush All Streams",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to flush all streams to the Postgres Warehouse before closing the connection.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "parallelism",
"aliases" : [ ],
"label" : "Parallelism",
"value" : "-1",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "parallelism_max",
"aliases" : [ ],
"label" : "Max Parallelism",
"value" : "16",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "default_target_schema_select_permission",
"aliases" : [ ],
"label" : "Default Target Schema Select Permission",
"options" : [ ],
"kind" : "STRING",
"description" : "The permission level required to select data from the default target schema.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "schema_mapping",
"aliases" : [ ],
"label" : "Schema Mapping",
"options" : [ ],
"kind" : "STRING",
"description" : "A mapping of source schema names to target schema names.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "add_metadata_columns",
"aliases" : [ ],
"label" : "Add Metadata Columns",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to add metadata columns to the target table.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "hard_delete",
"aliases" : [ ],
"label" : "Hard Delete",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to perform hard deletes when deleting data from the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "data_flattening_max_level",
"aliases" : [ ],
"label" : "Data Flattening Max Level",
"value" : "10",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum level of nested data structures to flatten when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "primary_key_required",
"aliases" : [ ],
"label" : "Primary Key Required",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not a primary key is required for the target table.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "validate_records",
"aliases" : [ ],
"label" : "Validate Records",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to validate records before writing them to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "temp_dir",
"aliases" : [ ],
"label" : "Temporary Directory",
"options" : [ ],
"kind" : "STRING",
"description" : "The directory to use for temporary files when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.host",
"aliases" : [ ],
"label" : "SSH Tunnel Host",
"options" : [ ],
"kind" : "STRING",
"description" : "Host of the bastion host, this is the host we'll connect to via ssh",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.port",
"aliases" : [ ],
"label" : "SSH Tunnel Port",
"value" : "22",
"options" : [ ],
"kind" : "INTEGER",
"description" : "Port to connect to bastion host",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.private_key",
"aliases" : [ ],
"label" : "SSH Tunnel Private Key",
"options" : [ ],
"kind" : "PASSWORD",
"description" : "A base64 encoded Private Key for authentication to the bastion host w/ key pair auth",
"hidden" : false,
"sensitive" : true,
"encoding" : "BASE64",
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.private_key_password",
"aliases" : [ ],
"label" : "SSH Tunnel Private Key Password",
"options" : [ ],
"kind" : "PASSWORD",
"description" : "Private Key Password, leave None if no password is set",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.username",
"aliases" : [ ],
"label" : "SSH Tunnel Username",
"options" : [ ],
"kind" : "STRING",
"description" : "Username to connect to bastion host",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.password",
"aliases" : [ ],
"label" : "SSH Tunnel Password",
"options" : [ ],
"kind" : "STRING",
"description" : "Password to connect to bastion host w/ basic auth",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.host_setting_name",
"aliases" : [ ],
"value" : "host",
"options" : [ ],
"kind" : "STRING",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.port_setting_name",
"aliases" : [ ],
"value" : "port",
"options" : [ ],
"kind" : "STRING",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.\n\n## Settings\n\n\n### User\n\nThe username used to connect to the Postgres Warehouse.\n\n### Password\n\nThe password used to authenticate the user.\n\n### Host\n\nThe hostname or IP address of the Postgres Warehouse server.\n\n### Port\n\nThe port number used to connect to the Postgres Warehouse server.\n\n### Database Name\n\nThe name of the database to connect to.\n\n### Default Target Schema\n\nThe default schema to use when writing data to the Postgres Warehouse.\n\n### State Schema\n\nThe default schema to use when writing Meltano state.\n\n### Batch Size Rows\n\nThe number of rows to write to the Postgres Warehouse in each batch.\n\n### Underscore Camel Case Fields\n\nWhether or not to convert field names from camel case to underscore-separated format.\n\n### Primary Key Required\n\nWhether or not a primary key is required for the target table.\n\n### Validate Records\n\nWhether or not to validate records before writing them to the Postgres Warehouse.\n\n### SSH Tunnel Host\n\nHost of the bastion host, this is the host we'll connect to via ssh\n\n### SSH Tunnel Port\n\nPort to connect to bastion host\n\n### SSH Tunnel Private Key\n\nA base64 encoded Private Key for authentication to the bastion host w/ key pair auth\n\n### SSH Tunnel Private Key Password\n\nPrivate Key Password, leave None if no password is set\n\n### SSH Tunnel Username\n\nUsername to connect to bastion host\n\n### SSH Tunnel Password\n\nPassword to connect to bastion host w/ basic auth",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31",
"type" : "DELETE"
}
}
}, {
"id" : "32ab6405-cfe1-4d12-a9fb-4639c5cebc7a",
"pluginType" : "TRANSFORMER",
"name" : "dbt",
"namespace" : "dbt",
"variant" : "dbt-labs",
"label" : "dbt",
"description" : " Power your project transformations with dbt™, a SQL-first transformation tool that enables analytics engineers to develop transformations with code.\n\n***Version Control and CI/CD***\n\nUse Matatika to deploy and promote changes between dev, UAT, and production environments.\n\n***Test and Document***\n\nUse Matatika to develop and test every model prior to production release, and share dynamically generated documentation with all stakeholders.\n\n***Develop***\n\nWrite modular data transformations in .sql – Matatika together with dbt handles the chore of dependency management. ",
"logoUrl" : "/assets/images/transformer/dbt.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/dbt/",
"pipUrl" : "dbt-core~=1.7.0 dbt-postgres~=1.7.0 dbt-snowflake~=1.7.0 dbt-bigquery~=1.7.0 dbt-sqlserver~=1.7.0 dbt-clickhouse~=1.7.0 certifi==2025.1.31",
"repo" : "https://github.com/dbt-labs/dbt-core",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "project_dir",
"aliases" : [ ],
"value" : "$MELTANO_PROJECT_ROOT/transform",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "profiles_dir",
"aliases" : [ ],
"value" : "$MELTANO_PROJECT_ROOT/transform/profile",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"env" : "DBT_PROFILES_DIR",
"configured" : false,
"protected" : false
}, {
"name" : "target",
"aliases" : [ ],
"value" : "$MELTANO_LOAD__DIALECT",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "source_schema",
"aliases" : [ ],
"value" : "$MELTANO_LOAD__TARGET_SCHEMA",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "target_schema",
"aliases" : [ ],
"value" : "analytics",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "target_path",
"aliases" : [ ],
"value" : "$MELTANO_PROJECT_ROOT/.meltano/transformers/dbt/target",
"options" : [ ],
"kind" : "STRING",
"description" : "Output path for dbt generated artifacts",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "models",
"aliases" : [ ],
"value" : "$MELTANO_TRANSFORM__PACKAGE_NAME $MELTANO_EXTRACTOR_NAMESPACE my_meltano_project",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "secret_git_credential",
"aliases" : [ ],
"options" : [ ],
"kind" : "STRING",
"description" : "A convenient and simple way to supply a git credential for packages resolve by dbt:deps",
"hidden" : false,
"sensitive" : true,
"env" : "DBT_ENV_SECRET_GIT_CREDENTIAL",
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : {
"docs-generate" : {
"args" : "docs generate",
"description" : "Generate documentation artifacts for your project."
},
"compile" : {
"args" : "compile",
"description" : "Generates executable SQL from source model, test, and analysis files. Compiled SQL files are written to the target/ directory."
},
"seed" : {
"args" : "seed",
"description" : "Load data from csv files into your data warehouse."
},
"test" : {
"args" : "test",
"description" : "Runs tests on data in deployed models."
},
"deps" : {
"args" : "deps",
"description" : "Pull the most recent version of the dependencies listed in packages.yml"
},
"run" : {
"args" : "run",
"description" : "Compile SQL and execute against the current target database."
},
"clean" : {
"args" : "clean",
"description" : "Delete all folders in the clean-targets list (usually the dbt_modules and target directories.)"
},
"snapshot" : {
"args" : "snapshot",
"description" : "Execute snapshots defined in your project."
}
},
"matatikaHidden" : false,
"requires" : [ {
"id" : "ea18593d-dae3-4d5a-9caf-255054b73860",
"pluginType" : "FILE",
"name" : "files-dbt",
"namespace" : "dbt",
"variant" : "matatika",
"description" : " Files dbt is a file bundle that automatically configures your project to run transforms with dbt.\nThe bundle includes template project configuration:\n\n- transform/models (directory)\n- transform/profile/profiles.yml\n- transform/dbt_project.yml\n- transform/.gitignore\n- transform/macros/centralize_test_failures.sql\n",
"hidden" : false,
"pipUrl" : "git+https://github.com/Matatika/files-dbt@v1.4",
"repo" : "https://github.com/Matatika/files-dbt",
"capabilities" : [ ],
"select" : [ ],
"update" : {
"transform/profile/profiles.yml" : "true"
},
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : " Files dbt is a file bundle that automatically configures your project to run transforms with dbt.\nThe bundle includes template project configuration:\n\n- transform/models (directory)\n- transform/profile/profiles.yml\n- transform/dbt_project.yml\n- transform/.gitignore\n- transform/macros/centralize_test_failures.sql\n"
} ],
"fullDescription" : " Power your project transformations with dbt™, a SQL-first transformation tool that enables analytics engineers to develop transformations with code.\n\n***Version Control and CI/CD***\n\nUse Matatika to deploy and promote changes between dev, UAT, and production environments.\n\n***Test and Document***\n\nUse Matatika to develop and test every model prior to production release, and share dynamically generated documentation with all stakeholders.\n\n***Develop***\n\nWrite modular data transformations in .sql – Matatika together with dbt handles the chore of dependency management. \n\n## Settings\n\n\n### target_path\n\nOutput path for dbt generated artifacts\n\n### secret_git_credential\n\nA convenient and simple way to supply a git credential for packages resolve by dbt:deps",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/32ab6405-cfe1-4d12-a9fb-4639c5cebc7a"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/32ab6405-cfe1-4d12-a9fb-4639c5cebc7a",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/32ab6405-cfe1-4d12-a9fb-4639c5cebc7a",
"type" : "DELETE"
}
}
}, {
"id" : "aae2e071-fd22-4e33-960e-62df01bc306f",
"pluginType" : "EXTRACTOR",
"name" : "tap-custom-test",
"variant" : "sit",
"label" : "Tap Custom Test",
"description" : "A dataplugin created during an SIT run",
"hidden" : false,
"pipUrl" : "git+https://github.com/Matatika/example-repository",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "username",
"aliases" : [ ],
"label" : "Username",
"options" : [ ],
"placeholder" : "username",
"kind" : "STRING",
"description" : "The username login credential.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "email",
"aliases" : [ ],
"label" : "Email",
"options" : [ ],
"placeholder" : "example@domain.com",
"kind" : "EMAIL",
"description" : "The email login credential.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"options" : [ ],
"placeholder" : "2020-01-01T00:00:00Z",
"kind" : "DATE_ISO8601",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "A dataplugin created during an SIT run\n\n## Settings\n\n\n### Username\n\nThe username login credential.\n\n### Email\n\nThe email login credential.\n\n### Start Date\n\nThe data to begin extracting data from, in ISO 8601 format.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f",
"type" : "DELETE"
}
}
}, {
"id" : "db429d46-1e3e-4068-a93b-e878645a4933",
"pluginType" : "EXTRACTOR",
"name" : "tap-test",
"variant" : "sit",
"hidden" : false,
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/db429d46-1e3e-4068-a93b-e878645a4933"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/db429d46-1e3e-4068-a93b-e878645a4933",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/db429d46-1e3e-4068-a93b-e878645a4933",
"type" : "DELETE"
}
}
}, {
"id" : "d4563e64-c8f5-4f89-a944-dd690d0b6b75",
"pluginType" : "LOADER",
"name" : "target-test",
"variant" : "sit",
"hidden" : false,
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/d4563e64-c8f5-4f89-a944-dd690d0b6b75"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/d4563e64-c8f5-4f89-a944-dd690d0b6b75",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/d4563e64-c8f5-4f89-a944-dd690d0b6b75",
"type" : "DELETE"
}
}
}, {
"id" : "e6c76c18-3236-4bf3-aa50-99351a595a9e",
"pluginType" : "TRANSFORM",
"name" : "dbt-tap-test",
"variant" : "sit",
"hidden" : false,
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/e6c76c18-3236-4bf3-aa50-99351a595a9e"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/e6c76c18-3236-4bf3-aa50-99351a595a9e",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/e6c76c18-3236-4bf3-aa50-99351a595a9e",
"type" : "DELETE"
}
}
}, {
"id" : "83764980-ad08-4296-9a1a-7b03cd0acfc8",
"pluginType" : "FILE",
"name" : "analyze-test",
"variant" : "sit",
"hidden" : false,
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/83764980-ad08-4296-9a1a-7b03cd0acfc8"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/83764980-ad08-4296-9a1a-7b03cd0acfc8",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/83764980-ad08-4296-9a1a-7b03cd0acfc8",
"type" : "DELETE"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins?page=0&size=20"
}
},
"page" : {
"size" : 20,
"totalElements" : 7,
"totalPages" : 1,
"number" : 0
}
}
View a dataplugin
GET /api/dataplugins/{dataplugin-id}
Returns the dataplugin {dataplugin-id}.
Prerequisites
- Dataplugin
{dataplugin-id}must exist
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://staging.meltano.com:443/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31' -i -X GET
import requests
url = "https://staging.meltano.com:443/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Dataplugin with HAL links.
{
"id" : "32374218-07a4-4a49-92f6-476339ae7b31",
"pluginType" : "LOADER",
"name" : "target-postgres",
"namespace" : "postgres_transferwise",
"variant" : "matatika",
"label" : "Postgres Warehouse",
"description" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.",
"logoUrl" : "/assets/logos/loaders/postgres.png",
"hidden" : false,
"docs" : "https://www-staging.matatika.com/data-details/target-postgres/",
"pipUrl" : "git+https://github.com/Matatika/pipelinewise-target-postgres.git@v0.2.1",
"repo" : "https://github.com/Matatika/pipelinewise-target-postgres",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "user",
"aliases" : [ "username" ],
"label" : "User",
"options" : [ ],
"kind" : "STRING",
"description" : "The username used to connect to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "password",
"aliases" : [ ],
"label" : "Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password used to authenticate the user.",
"hidden" : false,
"sensitive" : true,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "host",
"aliases" : [ "address" ],
"label" : "Host",
"options" : [ ],
"kind" : "STRING",
"description" : "The hostname or IP address of the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "port",
"aliases" : [ ],
"label" : "Port",
"value" : "5432",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The port number used to connect to the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "dbname",
"aliases" : [ "database" ],
"label" : "Database Name",
"options" : [ ],
"kind" : "STRING",
"description" : "The name of the database to connect to.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "default_target_schema",
"aliases" : [ ],
"label" : "Default Target Schema",
"value" : "analytics",
"options" : [ ],
"kind" : "STRING",
"description" : "The default schema to use when writing data to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "state_schema",
"aliases" : [ ],
"label" : "State Schema",
"value" : "public",
"options" : [ ],
"kind" : "STRING",
"description" : "The default schema to use when writing Meltano state.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"configured" : false,
"protected" : false
}, {
"name" : "ssl",
"aliases" : [ ],
"label" : "SSL",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to use SSL encryption when connecting to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false,
"value_post_processor" : "STRINGIFY"
}, {
"name" : "batch_size_rows",
"aliases" : [ ],
"label" : "Batch Size Rows",
"value" : "100000",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of rows to write to the Postgres Warehouse in each batch.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "underscore_camel_case_fields",
"aliases" : [ ],
"label" : "Underscore Camel Case Fields",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to convert field names from camel case to underscore-separated format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "flush_all_streams",
"aliases" : [ ],
"label" : "Flush All Streams",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to flush all streams to the Postgres Warehouse before closing the connection.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "parallelism",
"aliases" : [ ],
"label" : "Parallelism",
"value" : "-1",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "parallelism_max",
"aliases" : [ ],
"label" : "Max Parallelism",
"value" : "16",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "default_target_schema_select_permission",
"aliases" : [ ],
"label" : "Default Target Schema Select Permission",
"options" : [ ],
"kind" : "STRING",
"description" : "The permission level required to select data from the default target schema.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "schema_mapping",
"aliases" : [ ],
"label" : "Schema Mapping",
"options" : [ ],
"kind" : "STRING",
"description" : "A mapping of source schema names to target schema names.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "add_metadata_columns",
"aliases" : [ ],
"label" : "Add Metadata Columns",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to add metadata columns to the target table.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "hard_delete",
"aliases" : [ ],
"label" : "Hard Delete",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to perform hard deletes when deleting data from the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "data_flattening_max_level",
"aliases" : [ ],
"label" : "Data Flattening Max Level",
"value" : "10",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum level of nested data structures to flatten when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "primary_key_required",
"aliases" : [ ],
"label" : "Primary Key Required",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not a primary key is required for the target table.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "validate_records",
"aliases" : [ ],
"label" : "Validate Records",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to validate records before writing them to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "temp_dir",
"aliases" : [ ],
"label" : "Temporary Directory",
"options" : [ ],
"kind" : "STRING",
"description" : "The directory to use for temporary files when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.host",
"aliases" : [ ],
"label" : "SSH Tunnel Host",
"options" : [ ],
"kind" : "STRING",
"description" : "Host of the bastion host, this is the host we'll connect to via ssh",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.port",
"aliases" : [ ],
"label" : "SSH Tunnel Port",
"value" : "22",
"options" : [ ],
"kind" : "INTEGER",
"description" : "Port to connect to bastion host",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.private_key",
"aliases" : [ ],
"label" : "SSH Tunnel Private Key",
"options" : [ ],
"kind" : "PASSWORD",
"description" : "A base64 encoded Private Key for authentication to the bastion host w/ key pair auth",
"hidden" : false,
"sensitive" : true,
"encoding" : "BASE64",
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.private_key_password",
"aliases" : [ ],
"label" : "SSH Tunnel Private Key Password",
"options" : [ ],
"kind" : "PASSWORD",
"description" : "Private Key Password, leave None if no password is set",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.username",
"aliases" : [ ],
"label" : "SSH Tunnel Username",
"options" : [ ],
"kind" : "STRING",
"description" : "Username to connect to bastion host",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.password",
"aliases" : [ ],
"label" : "SSH Tunnel Password",
"options" : [ ],
"kind" : "STRING",
"description" : "Password to connect to bastion host w/ basic auth",
"hidden" : false,
"sensitive" : true,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.host_setting_name",
"aliases" : [ ],
"value" : "host",
"options" : [ ],
"kind" : "STRING",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "ssh_tunnel.port_setting_name",
"aliases" : [ ],
"value" : "port",
"options" : [ ],
"kind" : "STRING",
"hidden" : true,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.\n\n## Settings\n\n\n### User\n\nThe username used to connect to the Postgres Warehouse.\n\n### Password\n\nThe password used to authenticate the user.\n\n### Host\n\nThe hostname or IP address of the Postgres Warehouse server.\n\n### Port\n\nThe port number used to connect to the Postgres Warehouse server.\n\n### Database Name\n\nThe name of the database to connect to.\n\n### Default Target Schema\n\nThe default schema to use when writing data to the Postgres Warehouse.\n\n### State Schema\n\nThe default schema to use when writing Meltano state.\n\n### Batch Size Rows\n\nThe number of rows to write to the Postgres Warehouse in each batch.\n\n### Underscore Camel Case Fields\n\nWhether or not to convert field names from camel case to underscore-separated format.\n\n### Primary Key Required\n\nWhether or not a primary key is required for the target table.\n\n### Validate Records\n\nWhether or not to validate records before writing them to the Postgres Warehouse.\n\n### SSH Tunnel Host\n\nHost of the bastion host, this is the host we'll connect to via ssh\n\n### SSH Tunnel Port\n\nPort to connect to bastion host\n\n### SSH Tunnel Private Key\n\nA base64 encoded Private Key for authentication to the bastion host w/ key pair auth\n\n### SSH Tunnel Private Key Password\n\nPrivate Key Password, leave None if no password is set\n\n### SSH Tunnel Username\n\nUsername to connect to bastion host\n\n### SSH Tunnel Password\n\nPassword to connect to bastion host w/ basic auth",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/32374218-07a4-4a49-92f6-476339ae7b31",
"type" : "DELETE"
}
}
}
Initialise a new dataplugin
POST /api/dataplugins
Initialises a new dataplugin.
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins' -i -X POST \
-H 'Content-Type: application/json'
import requests
url = "https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("POST", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Dataplugin with HAL links.
{
"id" : "aae2e071-fd22-4e33-960e-62df01bc306f",
"hidden" : false,
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f"
},
"create dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f",
"type" : "PUT"
}
}
}
Create a dataplugin
PUT /api/dataplugins/{dataplugin-id}
Creates the dataplugin {dataplugin-id}.
Body
Dataplugin resource.
{
"name" : "tap-custom-test",
"variant" : "sit",
"pluginType" : "EXTRACTOR",
"label" : "Tap Custom Test",
"description" : "A dataplugin created during an SIT run",
"pipUrl" : "git+https://github.com/Matatika/example-repository",
"settings" : [ {
"name" : "username",
"kind" : "STRING",
"label" : "Username",
"placeholder" : "username",
"description" : "The username login credential.",
"protected" : false
}, {
"name" : "email",
"kind" : "EMAIL",
"label" : "Email",
"description" : "The email login credential.",
"placeholder" : "example@domain.com",
"protected" : false
}, {
"name" : "start_date",
"kind" : "DATE_ISO8601",
"label" : "Start Date",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"placeholder" : "2020-01-01T00:00:00Z",
"protected" : false
} ]
}
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f' -i -X PUT \
-H 'Content-Type: application/json' \
-d '{
"name" : "tap-custom-test",
"variant" : "sit",
"pluginType" : "EXTRACTOR",
"label" : "Tap Custom Test",
"description" : "A dataplugin created during an SIT run",
"pipUrl" : "git+https://github.com/Matatika/example-repository",
"settings" : [ {
"name" : "username",
"kind" : "STRING",
"label" : "Username",
"placeholder" : "username",
"description" : "The username login credential.",
"protected" : false
}, {
"name" : "email",
"kind" : "EMAIL",
"label" : "Email",
"description" : "The email login credential.",
"placeholder" : "example@domain.com",
"protected" : false
}, {
"name" : "start_date",
"kind" : "DATE_ISO8601",
"label" : "Start Date",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"placeholder" : "2020-01-01T00:00:00Z",
"protected" : false
} ]
}'
import requests
url = "https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f"
data = {
"name" : "tap-custom-test",
"variant" : "sit",
"pluginType" : "EXTRACTOR",
"label" : "Tap Custom Test",
"description" : "A dataplugin created during an SIT run",
"pipUrl" : "git+https://github.com/Matatika/example-repository",
"settings" : [ {
"name" : "username",
"kind" : "STRING",
"label" : "Username",
"placeholder" : "username",
"description" : "The username login credential.",
"protected" : false
}, {
"name" : "email",
"kind" : "EMAIL",
"label" : "Email",
"description" : "The email login credential.",
"placeholder" : "example@domain.com",
"protected" : false
}, {
"name" : "start_date",
"kind" : "DATE_ISO8601",
"label" : "Start Date",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"placeholder" : "2020-01-01T00:00:00Z",
"protected" : false
} ]
}
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers, data=data)
print(response.text.encode('utf8'))
Response
201 Created
Dataplugin with HAL links.
{
"id" : "aae2e071-fd22-4e33-960e-62df01bc306f",
"pluginType" : "EXTRACTOR",
"name" : "tap-custom-test",
"variant" : "sit",
"label" : "Tap Custom Test",
"description" : "A dataplugin updated during an SIT run",
"hidden" : false,
"pipUrl" : "git+https://github.com/Matatika/example-repository-2",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "username",
"aliases" : [ ],
"label" : "Username",
"placeholder" : "username",
"kind" : "STRING",
"description" : "The username login credential.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "email",
"aliases" : [ ],
"label" : "Email",
"placeholder" : "example@domain.com",
"kind" : "EMAIL",
"description" : "The email login credential.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"placeholder" : "2020-01-01T00:00:00Z",
"kind" : "DATE_ISO8601",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "A dataplugin updated during an SIT run\n\n## Settings\n\n\n### Username\n\nThe username login credential.\n\n### Email\n\nThe email login credential.\n\n### Start Date\n\nThe data to begin extracting data from, in ISO 8601 format.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f",
"type" : "DELETE"
}
}
}
Update a dataplugin
PUT /api/dataplugins/{dataplugin-id}
Updates the dataplugin {dataplugin-id}.
Prerequisites
- Dataplugin
{dataplugin-id}must exist
Body
Dataplugin resource.
{
"name" : "tap-custom-test",
"variant" : "sit",
"pluginType" : "EXTRACTOR",
"label" : "Tap Custom Test",
"description" : "A dataplugin updated during an SIT run",
"pipUrl" : "git+https://github.com/Matatika/example-repository-2",
"settings" : [ {
"name" : "username",
"kind" : "STRING",
"label" : "Username",
"placeholder" : "username",
"description" : "The username login credential.",
"protected" : false
}, {
"name" : "email",
"kind" : "EMAIL",
"label" : "Email",
"description" : "The email login credential.",
"placeholder" : "example@domain.com",
"protected" : false
}, {
"name" : "start_date",
"kind" : "DATE_ISO8601",
"label" : "Start Date",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"placeholder" : "2020-01-01T00:00:00Z",
"protected" : false
} ]
}
| Path | JSON Type | Format | Description | Constraints |
|---|---|---|---|---|
description | string | A description of the dataplugin | ||
repositoryUrl | string | A URL to the dataplugin repository | ||
settings | object[] | Array of Settings | The dataplugin settings |
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f' -i -X PUT \
-H 'Content-Type: application/json' \
-d '{
"name" : "tap-custom-test",
"variant" : "sit",
"pluginType" : "EXTRACTOR",
"label" : "Tap Custom Test",
"description" : "A dataplugin updated during an SIT run",
"pipUrl" : "git+https://github.com/Matatika/example-repository-2",
"settings" : [ {
"name" : "username",
"kind" : "STRING",
"label" : "Username",
"placeholder" : "username",
"description" : "The username login credential.",
"protected" : false
}, {
"name" : "email",
"kind" : "EMAIL",
"label" : "Email",
"description" : "The email login credential.",
"placeholder" : "example@domain.com",
"protected" : false
}, {
"name" : "start_date",
"kind" : "DATE_ISO8601",
"label" : "Start Date",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"placeholder" : "2020-01-01T00:00:00Z",
"protected" : false
} ]
}'
import requests
url = "https://staging.meltano.com:443/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f"
data = {
"name" : "tap-custom-test",
"variant" : "sit",
"pluginType" : "EXTRACTOR",
"label" : "Tap Custom Test",
"description" : "A dataplugin updated during an SIT run",
"pipUrl" : "git+https://github.com/Matatika/example-repository-2",
"settings" : [ {
"name" : "username",
"kind" : "STRING",
"label" : "Username",
"placeholder" : "username",
"description" : "The username login credential.",
"protected" : false
}, {
"name" : "email",
"kind" : "EMAIL",
"label" : "Email",
"description" : "The email login credential.",
"placeholder" : "example@domain.com",
"protected" : false
}, {
"name" : "start_date",
"kind" : "DATE_ISO8601",
"label" : "Start Date",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"placeholder" : "2020-01-01T00:00:00Z",
"protected" : false
} ]
}
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers, data=data)
print(response.text.encode('utf8'))
Response
200 OK
Dataplugin with HAL links.
{
"id" : "aae2e071-fd22-4e33-960e-62df01bc306f",
"pluginType" : "EXTRACTOR",
"name" : "tap-custom-test",
"variant" : "sit",
"label" : "Tap Custom Test",
"description" : "A dataplugin updated during an SIT run",
"hidden" : false,
"pipUrl" : "git+https://github.com/Matatika/example-repository-2",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "username",
"aliases" : [ ],
"label" : "Username",
"placeholder" : "username",
"kind" : "STRING",
"description" : "The username login credential.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "email",
"aliases" : [ ],
"label" : "Email",
"placeholder" : "example@domain.com",
"kind" : "EMAIL",
"description" : "The email login credential.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
}, {
"name" : "start_date",
"aliases" : [ ],
"label" : "Start Date",
"placeholder" : "2020-01-01T00:00:00Z",
"kind" : "DATE_ISO8601",
"description" : "The data to begin extracting data from, in ISO 8601 format.",
"hidden" : false,
"sensitive" : false,
"configured" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "A dataplugin updated during an SIT run\n\n## Settings\n\n\n### Username\n\nThe username login credential.\n\n### Email\n\nThe email login credential.\n\n### Start Date\n\nThe data to begin extracting data from, in ISO 8601 format.",
"_links" : {
"self" : {
"href" : "https://staging.meltano.com/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f"
},
"update dataplugin" : {
"href" : "https://staging.meltano.com/api/workspaces/080117cb-aea8-4d01-bc82-60d810ecdc66/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f",
"type" : "PUT"
},
"delete dataplugin" : {
"href" : "https://staging.meltano.com/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f",
"type" : "DELETE"
}
}
}
Delete a dataplugin
DELETE /api/dataplugins/{dataplugin-id}
Deletes the dataplugin {dataplugin-id}.
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://staging.meltano.com:443/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f' -i -X DELETE
import requests
url = "https://staging.meltano.com:443/api/dataplugins/aae2e071-fd22-4e33-960e-62df01bc306f"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("DELETE", url, headers=headers)
print(response.text.encode('utf8'))
Response
204 No Content
No response body provided.