> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2-docs-scavio-google-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Multimodal inputs

> Feed images, audio, video, and PDFs into any labeling or extraction agent.

Pass media through the matching `Agent.run()` argument and choose a model that supports the input modality. The `output_schema` pattern stays the same.

```python theme={null}
from typing import Literal

from agno.agent import Agent
from agno.media import Image
from agno.models.google import Gemini
from pydantic import BaseModel, Field


class Classification(BaseModel):
    label: Literal["wildlife", "landscape", "sports", "architecture", "other"] = Field(
        ..., description="The primary scene type of the image"
    )


agent = Agent(
    model=Gemini(id="gemini-3.5-flash"),
    instructions="You classify images by scene type.",
    output_schema=Classification,
)

url = "https://storage.googleapis.com/generativeai-downloads/images/generated_elephants_giraffes_zebras_sunset.jpg"
result = agent.run("Classify this image.", images=[Image(url=url)]).content
# Classification(label='wildlife')
```

## Input argument per modality

| Modality | Import                         | Argument                                    |
| -------- | ------------------------------ | ------------------------------------------- |
| Image    | `from agno.media import Image` | `images=[Image(url=...)]`                   |
| Audio    | `from agno.media import Audio` | `audio=[Audio(content=...)]`                |
| Video    | `from agno.media import Video` | `videos=[Video(content=..., format="mp4")]` |
| PDF      | `from agno.media import File`  | `files=[File(url=...)]`                     |

All four classes accept `url`, `filepath`, or raw bytes via `content`. The cookbook examples fetch audio and video bytes first. Agno installs `httpx`.

```python theme={null}
import httpx

from agno.agent import Agent
from agno.media import Audio
from agno.models.google import Gemini
from pydantic import BaseModel, Field


class Transcript(BaseModel):
    text: str = Field(..., description="Verbatim transcript of all spoken audio")


transcription_agent = Agent(
    model=Gemini(id="gemini-3.5-flash"),
    instructions="Transcribe all spoken audio. Return only the transcript.",
    output_schema=Transcript,
)

url = "https://agno-public.s3.us-east-1.amazonaws.com/demo_data/QA-01.mp3"
response = httpx.get(url, timeout=30.0)
response.raise_for_status()

result = transcription_agent.run(
    "Transcribe this audio.",
    audio=[Audio(content=response.content)],
).content
# Transcript(text='...')
```

## Bounding boxes

For region detection, return normalized coordinates so the result is resolution-independent.

```python theme={null}
from pydantic import BaseModel, Field


class BoundingBox(BaseModel):
    label: str = Field(..., description="What the box contains")
    x: float = Field(..., ge=0.0, le=1.0, description="Top-left x in [0, 1]")
    y: float = Field(..., ge=0.0, le=1.0, description="Top-left y in [0, 1]")
    width: float = Field(..., ge=0.0, le=1.0, description="Width in [0, 1]")
    height: float = Field(..., ge=0.0, le=1.0, description="Height in [0, 1]")
```

<Warning>
  State the `[0, 1]` coordinate system in both the field descriptions and the agent instructions. This keeps the four values consistent across images.
</Warning>

The [bounding boxes cookbook](https://github.com/agno-agi/agno/tree/main/cookbook/data_labeling/_08_image_bounding_boxes) has runnable single-object, multi-object, and per-box confidence variants.

## Transcription and diarization

Audio extraction covers transcription, speaker diarization, and timestamped segments. Each is a schema change over the same API.

| Output               | Schema shape                                               |
| -------------------- | ---------------------------------------------------------- |
| Flat transcript      | `{ text: str }`                                            |
| Speaker turns        | `{ turns: List[{ speaker, text }] }`                       |
| Timestamped segments | `{ segments: List[{ start_seconds, end_seconds, text }] }` |

## Model choice

The cookbook uses `gemini-3.5-flash` across its text, image, audio, video, and PDF recipes. A replacement model must support the input modality and structured output.

## Next steps

| Task                     | Guide                                                             |
| ------------------------ | ----------------------------------------------------------------- |
| Define the output schema | [Data extraction](/use-cases/data-labeling/structured-extraction) |
| Assign labels to media   | [Classification](/use-cases/data-labeling/classification)         |
| Review media labels      | [Quality pipeline](/use-cases/data-labeling/quality-pipeline)     |

## Developer Resources

* [Image extraction cookbook](https://github.com/agno-agi/agno/tree/main/cookbook/data_labeling/_07_image_extraction)
* [Audio transcription cookbook](https://github.com/agno-agi/agno/tree/main/cookbook/data_labeling/_11_audio_transcription)
* [Video extraction cookbook](https://github.com/agno-agi/agno/tree/main/cookbook/data_labeling/_14_video_extraction)
* [Document extraction cookbook](https://github.com/agno-agi/agno/tree/main/cookbook/data_labeling/_16_document_extraction)
* [Multimodal agents](/multimodal/overview)