Skip to content

Post-LLM Orchestration: Lar-JEPA

Lár isn't just for building LLM Chatbots. It is the deterministic execution spine for the next generation of AI research.

As the industry moves away from purely autoregressive models (like GPT-4) toward Predictive World Models (like Yann LeCun's JEPA architecture), researchers face a massive bottleneck: How do you orchestrate and test a model that outputs abstract mathematical tensors instead of English text?

Most orchestration frameworks (LangChain, AutoGPT) assume the agent's mind is a sequence of conversational text. They crash if you hand them a 768-dimensional NumPy array representing a "collision state."

This is why we built Lar-JEPA: a dedicated testbed and pattern library for World Model orchestration.

The "Trio Architecture"

The Lar-JEPA repository introduces researchers to the concept of The Trio Architecture:

  1. The World Simulator (JEPA): The "Imagination." This model predicts possible future states. It plans the best path to a goal by hallucinating consequences in a latent mathematical space, completely bypassing token-by-token text generation.
  2. The Execution Spine (Lár): Lár passes the abstract latent tensors (the JEPA predictions) through strict System 2 RouterNodes. It evaluates the mathematical danger or reward of a future state and reroutes the execution flow before the physical action occurs.
  3. The Cognitive Memory (DMN): The Default Mode Network provides episodic memory. When the agent "sleeps," the DMN scans the day's Lár execution logs and consolidates those slow, expensive JEPA simulations into fast, cheap, permanent "muscle memory" heuristics.

How to use Lár for World Models

If you are a researcher training a new predictive model, here is how you use Lár as your evaluation testbed without writing brittle while True loops.

1. Leverage Native Tensor Logging

The biggest headache of testing World Models is debugging the latent space. If you print a tensor to a console, it's unreadable. If you try to save it to a standard JSON audit log, the JSON stringifier crashes (TypeError: Object of type Tensor is not JSON serializable).

In the Lar-JEPA repository, our AuditLogger has been custom-patched with a TensorSafeEncoder.

You can now pass massive PyTorch or NumPy tensors natively in your Lár GraphState.

# The internal state can hold raw tensors securely
state.set("predicted_world_state", torch.randn(1, 768))

When the GraphExecutor logs the step, it safely intercepts the tensor and serializes a metadata reference to the JSON file: 

{
  "__type__": "Tensor/Array",
  "shape": [1, 768],
  "dtype": "float32"
}

2. Implement System 1 / System 2 Routing

You can use Lár's built-in RouterNode to formally test and measure the difference between fast-reflex execution (System 1) and deep-simulation planning (System 2).

Instead of parsing an LLM string (if "crash" in response:), your RouterNode evaluates the math directly:

def evaluate_danger(state: GraphState) -> str:
    # 1. Extract the raw tensor prediction from the JEPA
    prediction_tensor = state.get("jepe_prediction_tensor")

    # 2. Mathematically evaluate the danger threshold in latent space
    collision_probability = calculate_collision_vector(prediction_tensor)

    # 3. Route deterministically based on math, not text
    if collision_probability > 0.85:
        print("System 2 Intercept: Danger mathematically detected. Vetoing action.")
        return "REPLAN_NODE" # Veto and send back to imagination
    else:
        return "EXECUTE_NODE" # Safe, proceed to physical motors

Get Started

Instead of building your testing framework from scratch, clone the testbed:

  1. Visit the Lar-JEPA GitHub Repository
  2. Run 13_world_model_jepa.py to see a full abstract simulation loop in action.
  3. Replace the MockJEPA class with your actual PyTorch model.