As of WorkflowGen 10.2.0, the Configuration panel includes a new AI settings tab, available to WorkflowGen Administrators only.
This tab centralizes everything required to configure and monitor embeddings generation, from provider connectivity and chunking behavior to runtime status and token usage, providing you with one place to set it up and then monitor whether generation is healthy, failing, or burning tokens faster than expected.
Embeddings configuration
The Embeddings section lets you point WorkflowGen to the embedding API you want to use and control how requests are made.
Connection + authentication
- API provider
- API endpoint
- Authentication method
- Authentication header
- API key
Model + vector shape
- Model
- Dimensions
Throughput + resilience
- API call delay (ms)
- Maximum retries
- Retries delay (ms)
- Batch size
Chunking & language settings
Embeddings are generated from chunked text. The AI tab exposes the chunking knobs so you can tune recall, cost, and processing time:
- Chunk size
- Chunk overlap
- Maximum chunk count per dataset
- Chunk language code
- PostgreSQL text search language (PostgreSQL deployments only)
Monitoring: Embedding status, errors & token usage
The AI tab also gives you visibility into what’s happening at runtime:
Embedding status
- Total chunks
- Chunks with embeddings
- Pending chunks
- Failed chunks
- Total API tokens used (for embedding generation)
If WorkflowGen detects recent embedding errors, it shows a Last 10 errors table with:
- Dataset ID
- Chunk index
- Status code
- Error message
- Token usage
Token usage (all processes)
- Total API tokens used (across processes)
- Token usage per process
Quick tip
In many environments, the settings’ default values are a good starting point. In practice, the most common things you’ll want to customize are:
- Authentication (method/header/key)
- Model + dimensions (to match your provider/model requirements)
- Delays, batch size, and retries (to balance speed vs. rate limits / transient failures)
- Chunk sizing/overlap (to balance embedding quality vs. token consumption)
For more information, including default values, see the AI section in the WorkflowGen Administration Guide.