Task count and sizes Effective resource management is an important part of ensuring that Decodable can stream and process your data efficiently. The amount of resources, such as how much CPU and Memory to allocate for streaming and data processing, is determined by task size. In order to ensure that you have enough resources available for your use case, make sure that your task size is large enough to handle the types of processing that you want to do. It is also recommended to monitor the metrics and reprovision resources as required. Update task size Perform the following steps to update the task size of a connection or pipeline. Select the Connections or Pipelines page, and select the connection or pipeline that you’d like to adjust the task size for. The overview page for the selected resource opens. You can only change the task size when activating a connection or pipeline. If the connection or pipeline is currently running, stop it. Select Start to run it again. Select the task size that represents the amount of resources that you’d like to allocate to the connection or pipeline. Then, select Start. MongoDB connections must have a task size of M or larger. In addition, you cannot update the task size or count for the REST Connector and the Datagen Connector. Update the number of tasks Perform the following steps if you want to update the number of tasks for a specific connection or pipeline. If you are seeing a large number of “Maximum unconsumed records”, then we recommend increasing the number of tasks. Select the Connections or Pipelines page, and select the connection or pipeline that you’d like to adjust the task count for. The overview page for the selected resource opens. You can only change the number of tasks when activating a connection or pipeline. If the connection or pipeline is currently running, stop it. Select Start to run it again. Select the number of tasks that you’d like to allocate to the connection or pipeline. Then, select Start. Tasks that are size S can have a maximum of 1 task. Determining what task size you need When you specify a task size, you are specifying the maximum amount of resources that a task requires. In general, the task size that you need depends on the resource requirements of the use case you have. The resource requirements are mostly determined by the amount of data to process and the complexity of your pipeline. We generally recommend starting with task size M, unless you know it is a very basic or low-throughput pipeline or connection. If you experience stability issues with task size S, upgrade the task size to M. The M and L task sizes support task counts > 1. As a basic rule of thumb, the task size should correspond with the complexity of a pipeline, and the task count with the data volume. You can use the following table as a rough estimate for what task size you should use. Task size Use for… S Basic processing and transforms of data, such as filtering. "S" task sizes can have a maximum of 1 task. M Advanced processing and transforms of data, such as data aggregations. L Real-time streaming joins. This size is not available on the free plan.