ThermoQwen TSF

ThermoQwen TSF is an LLM-based thermocline depth time-series forecaster. It uses Qwen3-1.7B + LoRA as a regression model to infer future thermocline center depths from short local AUV observation segments. Each segment contains the past K local depth-temperature/salinity summaries and seasonal context. The model predicts the next H thermocline center depths z, where z is the depth at which the absolute temperature gradient is maximal.

This repository is packaged as a Hugging Face custom model. The exported weights include the Qwen backbone, LoRA weights, and the TSF regression head. This model follows the license of the Qwen3-1.7B base model.

Training

Argo float profiles — International Argo Program (https://argo.ucsd.edu), CC-BY.
Citation: Argo (2024). Argo GDAC. SEANOE. https://doi.org/10.17882/42182
NOAA data (NCEI Accession 0173210) — derived from WOD CTD/XBT profiles (1960–2017), Public Domain.
Citation: Chu, P.C., Fan, C. (2018). https://doi.org/10.25921/dgak-7a43

Subset used: 110–119°E, 9–19°N (South China Sea).

Evaluation

The evaluation set includes in-situ CTD measurements in addition to held-out splits of the public datasets above.

Model Structure

Backbone: Qwen3-1.7B
Fine-tuning: LoRA on q_proj, k_proj, v_proj, and o_proj
Pooling: [last_token_pool ; mean_pool]
Regression head: LayerNorm -> Linear(4096, 256) -> GELU -> Dropout -> Linear(256, 64) -> GELU -> Dropout -> Linear(64, H)
Output: future thermocline center depths in meters through outputs.pred_depths

Input Format

The user-facing input is a JSON-style local observation segment. Each element in observations is a compact summary of one local AUV observation window:

depth: AUV depth in meters, using negative values underwater.
T_grad: maximum absolute temperature difference inside the local sampled window.
avg_T: average temperature inside the local sampled window.
S_grad: maximum absolute salinity difference inside the local sampled window.
avg_S: average salinity inside the local sampled window.

{
  "season": "Winter",
  "doy": 7,
  "doy_sin": 0.1202,
  "doy_cos": 0.9927,
  "typical_thermocline_depth": "65-125 m",
  "horizon": 5,
  "observations": [
    {"depth": -109.7, "T_grad": 0.6064, "avg_T": 16.46, "S_grad": 0.0075, "avg_S": 34.61},
    {"depth": -174.3, "T_grad": 0.1081, "avg_T": 14.44, "S_grad": 0.0048, "avg_S": 34.55}
  ]
}

The JSON data is converted to the same Qwen chat-style prompt used during fine-tuning, then tokenized and passed to the custom model. A minimal example is shown below.

import json
import torch
from transformers import AutoModel, AutoTokenizer

model_id = "zetian123123/thermo-qwen3-tsf"

sample_json = """
{
  "season": "Winter",
  "doy": 7,
  "doy_sin": 0.1202,
  "doy_cos": 0.9927,
  "typical_thermocline_depth": "65-125 m",
  "horizon": 5,
  "observations": [
    {"depth": -109.7, "T_grad": 0.6064, "avg_T": 16.46, "S_grad": 0.0075, "avg_S": 34.61},
    {"depth": -174.3, "T_grad": 0.1081, "avg_T": 14.44, "S_grad": 0.0048, "avg_S": 34.55},
    {"depth": -33.6, "T_grad": 2.7128, "avg_T": 24.29, "S_grad": 0.4870, "avg_S": 34.07}
  ]
}
"""


def build_prompt(sample):
    season = sample["season"]
    horizon = int(sample["horizon"])
    obs_lines = []
    for idx, obs in enumerate(sample["observations"], start=1):
        obs_lines.append(
            f"Step {idx:2d}: depth={obs['depth']:7.1f}m"
            f"  T_grad={obs['T_grad']:.4f}  avg_T={obs['avg_T']:.2f}"
            f"  S_grad={obs['S_grad']:.4f}  avg_S={obs['avg_S']:.2f}"
        )

    return (
        f"<s><|im_start|>system\n"
        f"You are an expert oceanographer. In {season}, predict the thermocline "
        f"center depth (depth of maximum temperature gradient) for the next "
        f"{horizon} timesteps based on AUV observations.<|im_end|>\n"
        f"<|im_start|>user\n"
        f"Season: {season} | DOY: {int(sample['doy'])} | "
        f"sin={sample['doy_sin']:.4f} | cos={sample['doy_cos']:.4f}\n"
        f"In {season}, thermocline typically at "
        f"{sample['typical_thermocline_depth']}.\n\n"
        f"[AUV Observations - {len(sample['observations'])} steps]\n"
        + "\n".join(obs_lines)
        + f"\n\nPredict thermocline center depth for the next {horizon} timesteps."
        f"<|im_end|>\n"
        f"<|im_start|>assistant\n"
    )


tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True, torch_dtype="auto")
model.eval()

sample = json.loads(sample_json)
prompt = build_prompt(sample)
inputs = tokenizer(
    prompt,
    add_special_tokens=False,
    max_length=1024,
    padding="max_length",
    truncation=True,
    return_tensors="pt",
)

with torch.inference_mode():
    outputs = model(**inputs)

print(outputs.pred_depths.squeeze(0).tolist())

Output

The model returns:

outputs.logits: normalized depth predictions in [-1, 1]
outputs.pred_depths: denormalized depths in meters

Depth convention: water depth is represented as a negative value, e.g. -50 m.

Intended Use

This model is intended for offline thermocline depth forecasting research from AUV/CTD observation summaries. It is not a standalone navigation controller and should not be used for safety-critical AUV decisions without additional validation.