Discord-Micae-Hermes-3-3B — LoRA-merged conversational model (ChatML)

Model Description

Discord-Micae-Hermes-3-3B is a new finetune on NousResearch/Hermes-3-Llama-3.2-3B. This model serves as a foundation for our ongoing exploration into the capabilities of human-adjacent text generation.

Sample Conversation

User:

what do you think about ai?

Micae:

I'm not a fan of AI but I can understand why people are excited to use it. It's like the first time they got an electric car, or when they were able to fly in space, that excitement is real

User:

where do you think it's heading?

Micae:

It will be used for good and bad things just as with other techs

User:

what do you do use it for personally?

Micae:

Nothing much yet, I hope to find some stuff later on.

User:

do you like pokemon?

Micae:

Yes I love Pokémon.

User:

what's your favorite pokemon

Micae:

Garchomp

Training Details

• Fine-Tuning Method: LoRA merge (α = 32, r = 8 dropout = 0.1)
•
Training Schedule:
- 17M tokens of 260 thousand single-turn exchanges (STX) – 6 epochs @ 2e-5
- 5.5M tokens of 101 thousand multi-turn chains – 6 epochs @ 2e-5
- Combined dataset – 1 epoch @ 1e-5
• Scheduler: Cosine schedule with 220 warmup steps per phase
• Batching: Effective size of 126 (7 per device × 18 gradient accumulation steps)

Training took place over 17 days on a single GTX 1080 (8GB).

Dataset

The model was fine-tuned on the mookiezi/Discord-OpenMicae dataset.

Intended Use

• Conversational AI research
• Experimentation with dialogue agents trained on Discord data
• Chatbots requiring casual, human-like tone

Limitations

• The model inherits potential biases from Discord-style language.
• It is not safety-aligned for deployment without moderation.
• While it does inherit knowledge from Hermes-3-3B it is not intended for factual or sensitive information retrieval.

Usage example (Transformers pipeline)

from transformers import pipeline

gen = pipeline("text-generation", model="mookiezi/Discord-Micae-Hermes-3-3B")
print(gen(
    "<|im_start|>user\nwhat do you do?<|im_end|>\n<|im_start|>assistant\n",
    max_new_tokens=100
))