Profiling API

This reference covers profiling task requests, responses, and states. For a first collection, see the Profiling Guide.

huatuo-apiserver creates, supervises, and stops profiling tasks through /v1/profiling. Each HTTP creation request is an independent execution. Profiling tasks are persisted as Jobs in huatuo-apiserver; the node keeps only in-memory operations and profiler processes.

Task control does not depend on Elasticsearch. Raw-result queries and dashboard links require it. huatuo-bamai and huatuo-apiserver must use the same index.

Request Conventions

huatuo-apiserver listens on :12740 by default. These examples set the address and Bearer token through environment variables:

API_BASE="http://127.0.0.1:12740"
API_TOKEN="REPLACE_WITH_RANDOM_HEX"

Every request must supply the configured Bearer token in Authorization:

Authorization: Bearer REPLACE_WITH_RANDOM_HEX

Non-admin users need permissions for /v1/profiling and /v1/profiling/**. Permissions may include an HTTP method prefix, such as GET /v1/profiling/**. Successful responses contain only data:

{
  "data": {}
}

Error responses contain a stable error code and a readable message:

{
  "error": {
    "code": "result_unavailable",
    "message": "Job state does not provide a complete Profiling result"
  }
}

Clients should branch on error.code, not parse error.message.

Query Capabilities

Before creating a task, query supported profiling types, languages, CPU and memory modes, and runtime parameters:

curl -sS \
  -H "Authorization: Bearer ${API_TOKEN}" \
  "${API_BASE}/v1/profiling/capabilities"

data.items is a static capability table shipped with each version. Entries contain:

CPU profiling for c, c++, and go supports oncpu and offcpu; java and python support only oncpu. Memory profiling supports:

Create a Task

JSON parameters for POST /v1/profiling:

Every request creates a new task, even with identical content. Exceeding per-node or process-wide active task quotas returns HTTP 429.

Create a host Go CPU profiling task:

curl -sS -i \
  -X POST \
  -H "Authorization: Bearer ${API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "cpu",
    "language": "go",
    "mode": "oncpu",
    "scope": "host",
    "duration_seconds": 60,
    "hostname": "node-01"
  }' \
  "${API_BASE}/v1/profiling"

Create a Java live-object profiling task in a container:

curl -sS -i \
  -X POST \
  -H "Authorization: Bearer ${API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "memory",
    "language": "java",
    "mode": "object_usage",
    "scope": "container",
    "duration_seconds": 60,
    "container_id": "9f4c2f1a8b7d",
    "hostname": "node-01"
  }' \
  "${API_BASE}/v1/profiling"

Success returns 201 Created, a Location header pointing to the new task, and the task ID needed for later queries:

{
  "data": {
    "request_id": "<profile-job-id>",
    "hostname": "node-01",
    "duration_seconds": 60,
    "scope": "host",
    "type": "cpu",
    "language": "go",
    "mode": "oncpu",
    "status": "pending",
    "created_at": "2026-08-24T10:00:00Z",
    "updated_at": "2026-08-24T10:00:00Z"
  }
}

JOB_ID="<profile-job-id>"

Query Tasks

Task list

GET /v1/profiling supports these query parameters:

Query the 20 most recent profiling tasks:

curl -sS -G \
  -H "Authorization: Bearer ${API_TOKEN}" \
  --data-urlencode "limit=20" \
  --data-urlencode "offset=0" \
  "${API_BASE}/v1/profiling"

The server returns only profiling tasks, ordered by created_at descending. data.items is the task array; data.limit and data.offset are the effective pagination parameters; data.has_more indicates another page. Non-admin users see only their own tasks.

Task details

GET /v1/profiling/:request_id returns one task:

curl -sS \
  -H "Authorization: Bearer ${API_TOKEN}" \
  "${API_BASE}/v1/profiling/${JOB_ID}"

Task information is in data:

Task states:

terminal.outcome is completed, failed, stopped, or unknown. For failed, inspect terminal.reason and terminal.message. stopped means an explicit stop and possibly incomplete results. unknown means huatuo-apiserver cannot confirm the final operation state, although durably published results may still be available.

operation_lost and execution_timed_out are terminal.reason codes, not task states. Other reasons include startup or stop timeouts, unavailable nodes, execution-capacity limits, startup or runtime failure, invalid_node_request for an invalid node request constructed by huatuo-apiserver, and protocol errors.

Get Results

GET /v1/profiling/:request_id/raw returns raw windows for tasks with terminal.outcome of completed or unknown only while the durable publication marker exists. A missing or expired marker returns result_not_found. Active states return result_not_ready; failed and stopped return result_unavailable.

Responses may be large; save them directly to a file:

curl -sS \
  -H "Authorization: Bearer ${API_TOKEN}" \
  -o profile-raw.json \
  "${API_BASE}/v1/profiling/${JOB_ID}/raw?limit=100&offset=0"

Windows are in data.items; data.limit, data.offset, and data.has_more describe pagination. Each record includes uploaded_timestamp, started_timestamp, profile_type, and pprof-compatible profile data. A durably published empty result succeeds with an empty items array. Explicit limit=0 is invalid; only omission uses the default. limit defaults to 20 and is capped at 100. Encoded Profile data above 64 MiB returns 413 result_too_large; reduce limit and retry.

Flame graphs read and merge batches of 100 windows, with a total limit of 100000 per request. At one window per second for an hour, all 3600 windows participate. Exceeding 100000 returns an error asking for a narrower time range or filters. Query or merge failures never return partial flame graphs. Raw Profile lists remain scoped to a task; their page limit does not cap aggregation for a complete flame graph.

Stop a Task

Stopping is an asynchronous intent accepted for tasks in pending, running, or stopping:

curl -sS \
  -X POST \
  -H "Authorization: Bearer ${API_TOKEN}" \
  "${API_BASE}/v1/profiling/${JOB_ID}/stop"

Success returns 200 OK with the current task snapshot. Terminal tasks return 409 Conflict. Because results may be incomplete, stopped tasks do not support result queries.

Unified flame graph queries

Grafana uses the Pyroscope-compatible Protobuf endpoints under /v1/profiling/flamegraph/. SelectMergeStacktraces selectors require scope="host" or scope="container". region is the Agent’s configured region; hostname is only the collection node. Container queries require container_id or one task id. Host queries cannot include container filters. For example: {region="example",hostname="node-01",scope="container",container_id="…",id="…"}.

Each label accepts one equality condition. On this endpoint the wildcard values *, all, and the empty string only drop the restriction for that label, so {scope="host",hostname="*",id="job-1"} still selects one task; scope is still required. Repeating a label with the same value is accepted, while contradictory duplicates such as {hostname="host-a",hostname="host-b"} are rejected with 400 and the error reports both values, because one query cannot select two hosts. A matcher that uses any operator other than = is rejected as well, including hostname!="*".

The shared dashboard UID is continuous-profiling; variables are ordered as Region, node, scope, container, profile type, and task. The timeline and flame graph use the same task filter. With All containers selected, the task still identifies one target. Physical usage always requires one task and selects only the latest snapshot. The task summary uses the Infinity data source in pyroscope.yaml, with the same API token as the Pyroscope data source. result_url includes the stored windows’ Region and actual profile type. Results without windows do not provide a dashboard link.