FabledCurator/frontend/src/components/settings/SystemHealthTab.vue

<template>
  <!-- A Settings tab, not a page of its own (operator 2026-09-02): the first
       cut hung this off the health dot alone, which is a target you have to
       already suspect something to look for. Settings is where someone goes to
       ask the instance about itself.

       ONE table, not two panes (operator 2026-09-23: "I feel that we can
       probably combine the two sections into a single table and to format it
       in such a way that it appears more bounded and less free-form or open").

       They were the roster (milestone 365, what is running) above the worker
       lanes (milestone 422, how hard) — and the four celery lanes appeared in
       BOTH, once as a status row and again as a control row. The page said
       "ML tagging is running" and then, four hundred pixels later, "ML tagging
       1 / 1 busy". Two answers to one question, from two endpoints, free to
       disagree with each other on screen.

       So it is one row per part now, and a lane's row simply has controls
       where a datastore's has none. -->
  <div>
    <div class="d-flex align-center mb-1">
      <v-spacer />
      <span class="fc-sys__checked">
        {{ store.checkedAt ? `checked ${formatRelative(store.checkedAt)}` : 'checking…' }}
      </span>
    </div>

    <p class="fc-sys__lede text-body-2 mb-4">
      Every moving part of FabledCurator: whether it is still checking in, and —
      for the worker lanes — how much work it is allowed to take at once. Parts
      are learned as they appear, so anything that has run at least once stays
      listed. That is what lets a stopped one be noticed rather than simply
      vanishing.
    </p>

    <v-alert
      v-if="store.lastError" type="error" variant="tonal" density="compact"
      class="mb-4"
    >
      Could not reach FabledCurator: {{ store.lastError }}
    </v-alert>

    <v-alert
      v-if="notice"
      :type="notice.type" variant="tonal" density="compact"
      class="mb-4" closable
      @click:close="notice = null"
    >
      {{ notice.text }}
    </v-alert>

    <v-card v-if="!store.lastError" variant="flat" class="fc-parts">
      <v-table density="compact" class="fc-parts__table">
        <thead>
          <tr>
            <th>Part</th>
            <th>Queues</th>
            <th class="text-right">Waiting</th>
            <th class="text-right">Workers</th>
            <th class="text-center" style="width: 160px;">Max workers</th>
          </tr>
        </thead>
        <tbody>
          <tr v-if="!rows.length">
            <td colspan="6" class="text-center py-6 fc-sys__muted">
              Still gathering — this fills in on the first check.
            </td>
          </tr>

          <tr v-for="row in rows" :key="row.key">
            <td>
              <div class="fc-parts__name">
                <span class="fc-sys__dot" :class="`fc-sys__dot--${row.state}`" />
                {{ row.name }}
                <span class="fc-sys__kind">{{ row.kindLabel }}</span>
              </div>
              <!-- The sentence, not just a chip. At the moment someone is
                   deciding whether to go and open Portainer, "has not checked
                   in for 6 min" is the thing that answers them. -->
              <div class="fc-parts__detail">{{ row.detail }}</div>
              <!-- The operator asked for a lane held at full for a long time to
                   TRIGGER growth. It reports instead: extra slots do not make a
                   long task finish sooner, so scaling on this would spend
                   memory to change nothing. Seeing a lane wedged on one slow
                   job is the useful half. -->
              <div v-if="row.stuckFor" class="fc-parts__detail text-warning">
                all workers busy for {{ row.stuckFor }}
              </div>
              <!-- The nudge. Operator: "there needs to be something that tells
                   the user to bump those numbers to improve processing rate or
                   they'd never know the controls exist." It fires only when
                   raising the cap would actually help — see `laneAdvice`. -->
              <div v-if="row.advice" class="fc-parts__advice">
                <v-icon size="14" class="mr-1">mdi-arrow-up-bold-outline</v-icon>
                {{ row.advice }}
              </div>
            </td>

            <td class="fc-parts__queues">{{ row.queues || '—' }}</td>

            <!-- depth + reserved. LLEN alone reads 0 while a worker holds
                 prefetched tasks in memory, which is the number that would make
                 someone think a buried lane was idle. -->
            <td class="text-right fc-parts__num">
              {{ row.lane ? (row.lane.pending ?? '—') : '—' }}
            </td>

            <!-- Running, and how many of them are busy. Both are
                 MEASUREMENTS — the sizing pass owns this number, nobody sets
                 it — which is why neither is a control. -->
            <td class="text-right fc-parts__num">
              <template v-if="row.lane && row.lane.live.present">
                {{ row.lane.live.active }} / {{ row.lane.live.pool ?? '?' }}
              </template>
              <template v-else>—</template>
            </td>

            <td class="text-center">
              <template v-if="row.lane">
                <div class="d-flex align-center justify-center">
                  <v-btn
                    icon="mdi-minus" size="x-small" variant="text"
                    :disabled="busy === row.lane.name || row.lane.slots_cap <= 0"
                    :aria-label="`Allow ${row.name} fewer workers`"
                    @click="step(row.lane, -1)"
                  />
                  <span class="fc-parts__slots">{{ row.lane.slots_cap }}</span>
                  <v-btn
                    icon="mdi-plus" size="x-small" variant="text"
                    :disabled="busy === row.lane.name
                      || row.lane.slots_cap >= row.lane.ceiling"
                    :aria-label="`Allow ${row.name} more workers`"
                    @click="step(row.lane, 1)"
                  />
                </div>
                <!-- The REASON a higher number is unavailable, always on
                     screen — and, just as importantly, the headroom that says
                     raising it is even possible. A greyed control with no
                     explanation reads as a bug. -->
                <div class="fc-parts__sub">
                  {{ row.lane.ceiling === 0
                    ? 'no room on this machine'
                    : `up to ${row.lane.ceiling}` }}
                </div>
              </template>
              <span v-else class="fc-sys__muted">—</span>
            </td>
          </tr>
        </tbody>
      </v-table>
    </v-card>

    <!-- Where the numbers came from, and when.
         The worker counts are read by a background sweep and stored, not
         fetched when this page loads (operator, 2026-09-23: *"is there a
         reason this info isn't being tracked in the background and stored in
         some way?"*). They are therefore up to one sweep old, and saying so
         is the difference between a lagging number and a wrong one. -->
    <p v-if="!store.lastError" class="fc-parts__measured mt-2 mb-0">
      <template v-if="measured">
        Waiting and worker counts were measured {{ formatRelative(measured) }}<span
          v-if="sweepSeconds"
        >, and are re-read every {{ Math.round(sweepSeconds) }}s</span>.
      </template>
      <template v-else>
        Waiting and worker counts have not been measured yet — the first
        reading lands within a minute of startup.
      </template>
    </p>

    <!-- What the one control actually means. Operator, 2026-09-23: *"there's
         nothing to describe what 'auto' means or why their needs to be or
         should be on/off toggles."* There were three controls and no sentence
         explaining any of them; there is one now, and it is explained. -->
    <div class="fc-parts__legend mt-4">
      <p class="mb-2">
        <strong>Max workers</strong> is the only thing you set. How many
        workers a lane is actually running at any moment is decided for you —
        it grows to meet a backlog and gives the workers back when the queue
        empties, so an idle instance settles down to one of each without being
        told to. The cap is the ceiling on that, never a target, so raising it
        costs nothing until there is work to spend it on.
      </p>
      <p class="mb-2">
        <strong>A cap of zero turns the lane off.</strong> It keeps one parked
        process and stops taking work, so it stays listed here rather than
        looking like a crash. Three of the four lanes need to be running for
        FabledCurator to work at all; ML tagging is the one that is genuinely
        optional, and it ships at zero because switching it on downloads a
        model. <em>Up to N</em> beneath each dial is what this machine can
        hold — processor cores for every lane, and for ML tagging whichever
        runs out first, its cores or its memory. It is recalculated from the
        container's real limits every time this page loads.
      </p>
      <p class="mb-2">
        The shipped caps are one of each, which is right for a first boot and
        wrong for a busy library. A lane that is running everything its cap
        allows while work piles up <strong>will say so in its row</strong>, and
        raising that cap is the answer when it does.
      </p>
      <p v-if="store.thresholds" class="mb-0">
        A part is called <strong>stale</strong> after
        {{ Math.round(store.thresholds.stale_after_seconds / 60) }} min without a
        check-in and treated as <strong>stopped</strong> after
        {{ Math.round(store.thresholds.down_after_seconds / 60) }} min. The window
        is deliberately wide: a rolling deploy briefly runs two of a service and
        then neither, and an indicator that reddened on every update would stop
        being read.
      </p>
    </div>

    <!-- What an optional, off lane will cost BEFORE it is given a slot. Every
         number comes from the lane payload, and `measured` travels with them —
         an estimate is labelled rather than rounded into something that reads
         like a fact. -->
    <v-alert
      v-for="lane in offOptionalLanes" :key="`advisory-${lane.name}`"
      type="info" variant="tonal" density="compact" class="mt-4"
    >
      <div class="font-weight-medium mb-1">
        {{ lane.display_name }} is optional and currently off
      </div>
      <p class="mb-2">
        It computes image embeddings on the CPU — what similarity search,
        duplicate grouping and tag suggestions are built on. You only need it if
        you are <em>not</em> running the GPU agent, which does the same work
        faster.
      </p>
      <p class="mb-1">Raising its cap above zero downloads, once:</p>
      <ul class="mb-2">
        <li v-for="m in lane.models" :key="m.repo">
          <code>{{ m.repo }}</code> —
          {{ approx(m.measured) }}{{ gb(m.download_bytes) }} to download, and
          about {{ gb(m.resident_bytes) }} of RAM for
          <strong>each</strong> slot while it runs.
        </li>
      </ul>
      <p class="mb-0 text-caption">
        Each worker loads its own copy and asks for several processor cores
        while it runs, which is why this machine allows it at most
        {{ lane.ceiling }}.
        <template v-if="lane.ceiling === 0">
          It has too little memory to run this at all.
        </template>
      </p>
    </v-alert>
  </div>
</template>

<script setup>
import { computed, onMounted, onUnmounted, ref } from 'vue'

import { laneStuckFor, useSystemActivityStore } from '../../stores/systemActivity.js'
import { useSystemHealthStore } from '../../stores/systemHealth.js'
import { formatRelative } from '../../utils/date.js'
import { laneAdvice, measuredAt, mergeParts } from '../../utils/systemParts.js'

const store = useSystemHealthStore()
const lanesStore = useSystemActivityStore()

// Which lane is mid-write. Per-lane rather than a global flag so adjusting one
// lane does not freeze the others.
const busy = ref(null)
const notice = ref(null)

// The roster half needs no timer: TopNav already polls that same store every
// 15s for the health dot, on every route this tab is reachable from. The LANES
// have no such poller, so this tab owns them — 15s and `document.hidden`
// guarded, because `/api/system/workers` costs a broker round trip and
// v-window keeps a visited tab MOUNTED, so a naive interval would keep firing
// behind Maintenance and behind a backgrounded browser tab.
const LANE_POLL_MS = 15000
let pollId = null

onMounted(() => {
  store.refresh()
  lanesStore.loadLanes()
  pollId = setInterval(() => {
    if (document.hidden) return
    lanesStore.loadLanes()
  }, LANE_POLL_MS)
})
onUnmounted(() => { clearInterval(pollId) })

// One row per moving part, lanes joined to their roster entry. The join lives
// in `utils/systemParts.js` because its failure is silent: a lane that stops
// matching its part does not throw, it grows a SECOND row for the same worker
// — the duplication this table was built to remove, returning through the code
// that removed it.
const rows = computed(() => mergeParts(
  store.parts, lanesStore.lanes?.lanes ?? [], laneStuckFor,
).map((row) => ({ ...row, advice: laneAdvice(row.lane) })))

// When the stored worker numbers were last read, and how often that happens.
// Both come from the endpoint rather than being restated here — the cadence
// is `SWEEP_PERIOD_SECONDS`, and a second copy of it in the UI would be free
// to drift from the schedule it is describing.
const measured = computed(() => measuredAt(lanesStore.lanes?.lanes ?? []))
const sweepSeconds = computed(() => lanesStore.lanes?.sweep_period_seconds ?? null)

const offOptionalLanes = computed(() =>
  (lanesStore.lanes?.lanes ?? []).filter(
    (l) => l.optional && l.slots_cap === 0 && l.models?.length,
  ),
)

function gb(bytes) {
  return `${(bytes / 1024 ** 3).toFixed(1)} GB`
}

// An unmeasured figure says so. It decides whether someone's server survives,
// and presenting an estimate as a measurement is the failure this guards.
function approx(measured) {
  return measured ? '' : 'about '
}

async function apply(lane, fields) {
  busy.value = lane.name
  notice.value = null
  try {
    const reply = await lanesStore.setLane(lane.name, fields)
    if (reply && reply.fetching_models) {
      notice.value = {
        type: 'info',
        text: `${lane.display_name} is on. Downloading its model now — watch `
          + 'progress under Activity. It only happens once.',
      }
    }
    // There is no "saved but not applied" message any more, because there is
    // nothing to report yet: the endpoint answers once the cap is STORED and
    // tells the running lane afterwards, so the press cannot wait on a broker
    // round trip (operator: "we shouldn't have to wait for the validation
    // live"). Whether the lane has caught up is what its row says, and the
    // row updates on the next poll a few seconds later.
  } catch (e) {
    // The endpoint's `detail` is written to be read by a person ("cap 10000 is
    // above what this container can hold"). Surface it rather than a status
    // code, and never swallow it — a control that silently does nothing is
    // worse than one that refuses out loud.
    //
    // `e.body`, not `e.detail`: ApiError puts the parsed response on `.body`
    // and sets `.message` to the short `error` key.
    notice.value = { type: 'error', text: e.body?.detail || e.message }
  } finally {
    busy.value = null
  }
}

// The cap is the only thing an operator sets, and it doubles as the switch:
// zero means no consumers. How many workers actually run is the sizing pass's
// business — always on, reading the live pool every minute — so nothing here
// sends a worker count.
function step(lane, delta) {
  return apply(lane, { slots_cap: lane.slots_cap + delta })
}
</script>

<style scoped>
/* These lived in app.css for one commit, globalised so the roster and the
   worker lanes could share them. Merging the two into one table left a single
   consumer, so they come back here — a global utility with one caller is a
   rule nobody can find from the markup that uses it. */
.fc-sys__lede, .fc-sys__muted, .fc-sys__checked {
  color: rgb(var(--v-theme-on-surface) / 0.66);
}
.fc-sys__checked { font-size: 0.78rem; }

.fc-sys__dot { width: 9px; height: 9px; border-radius: 50%; flex: 0 0 auto; }
.fc-sys__dot--ok      { background: rgb(var(--v-theme-success)); }
.fc-sys__dot--stale   { background: rgb(var(--v-theme-warning)); }
.fc-sys__dot--degraded { background: rgb(var(--v-theme-warning)); }
.fc-sys__dot--down    { background: rgb(var(--v-theme-error)); }
.fc-sys__dot--unknown { background: rgb(var(--v-theme-on-surface) / 0.35); }

.fc-sys__kind {
  font-weight: 400; font-size: 0.72rem; text-transform: uppercase;
  letter-spacing: 0.04em; color: rgb(var(--v-theme-on-surface) / 0.5);
}

/* Bounded rather than free-form (operator 2026-09-23). What was here before was
   dotted rows floating on the page background with no header and no edge, so
   nothing said where the list began or what any column meant. */
.fc-parts {
  background: rgb(var(--v-theme-on-surface) / 0.04);
  border: 1px solid rgb(var(--v-theme-on-surface) / 0.12);
  border-radius: 6px;
  overflow: hidden;
}
.fc-parts__table :deep(thead th) {
  font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.05em;
  color: rgb(var(--v-theme-on-surface) / 0.55);
  border-bottom: 1px solid rgb(var(--v-theme-on-surface) / 0.18) !important;
  white-space: nowrap;
}
.fc-parts__table :deep(tbody td) {
  border-bottom: 1px solid rgb(var(--v-theme-on-surface) / 0.08) !important;
  padding-top: 8px; padding-bottom: 8px;
}
.fc-parts__table :deep(tbody tr:last-child td) { border-bottom: 0 !important; }

.fc-parts__name {
  display: flex; align-items: center; gap: 8px; font-weight: 600;
}
/* Clears the dot, so the sentence lines up under the name rather than under
   the indicator. */
.fc-parts__detail {
  font-size: 0.8rem; color: rgb(var(--v-theme-on-surface) / 0.72);
  padding-left: 17px;
}
.fc-parts__advice {
  font-size: 0.8rem; padding-left: 17px; margin-top: 2px;
  color: rgb(var(--v-theme-accent));
}
.fc-parts__queues {
  font-size: 0.78rem; color: rgb(var(--v-theme-on-surface) / 0.6);
}
.fc-parts__num { font-variant-numeric: tabular-nums; }
.fc-parts__measured {
  font-size: 0.78rem; color: rgb(var(--v-theme-on-surface) / 0.55);
}
.fc-parts__sub {
  font-size: 0.68rem; text-transform: uppercase; letter-spacing: 0.04em;
  color: rgb(var(--v-theme-on-surface) / 0.5); white-space: nowrap;
}

/* Fixed width so the number does not shift the +/- buttons as it changes
   between one and two digits. */
.fc-parts__slots {
  display: inline-block; min-width: 1.75em; text-align: center;
  font-variant-numeric: tabular-nums;
}

.fc-parts__legend {
  font-size: 0.82rem; line-height: 1.55;
  color: rgb(var(--v-theme-on-surface) / 0.66);
}
</style>