# Hugging Face Inference Client

This guide demonstrates how to interact with open-source LLMs hosted on the Hugging Face Hub using the `huggingface_hub` library. This allows you to use models like Mistral, Falcon, or Zephyr via API without downloading them locally.

**Modules Used:**
*   `huggingface_hub`: The official Python client for the Hugging Face Hub.
*   [[programming/python/modules/python-dotenv-module|python-dotenv]]: To manage the API token.
*   [[programming/python/modules/argparse-module|argparse]]: To handle command-line arguments.

## Prerequisites

1.  **Hugging Face Account**: Sign up at huggingface.co.
2.  **Access Token**: Go to Settings -> Access Tokens and create a new token (Read permissions).

## Installation

```bash
pip install huggingface_hub python-dotenv
```

## Setup

Create a `.env` file:
```text
HF_TOKEN=hf_your_token_here
```

## The Code

Save this as `hf_chat.py`.

```python
import os
import argparse
from dotenv import load_dotenv
from huggingface_hub import InferenceClient

# Load Config
load_dotenv()
TOKEN = os.getenv("HF_TOKEN")

def chat_with_model(model_id, prompt):
    if not TOKEN:
        print("Error: HF_TOKEN not found in .env file.")
        return

    print(f"Connecting to {model_id}...")
    
    try:
        client = InferenceClient(token=TOKEN)
        
        # Generate stream
        stream = client.text_generation(
            model=model_id,
            prompt=f"<s>[INST] {prompt} [/INST]", # Mistral/Llama formatting
            max_new_tokens=500,
            stream=True
        )

        print("\nResponse:")
        for response in stream:
            print(response, end="", flush=True)
        print()

    except Exception as e:
        print(f"Error: {e}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Hugging Face Inference Chat")
    parser.add_argument("prompt", help="The prompt to send")
    parser.add_argument("--model", default="mistralai/Mistral-7B-Instruct-v0.2", help="Model ID on HF Hub")
    
    args = parser.parse_args()
    
    chat_with_model(args.model, args.prompt)
```

## Usage

```bash
python hf_chat.py "Explain the theory of relativity simply."
```

[[programming/python/python]]