Samantha-1.2-Mistral-7B: Best LLM Trained on Philosophy, Psychology, and Personal Relationships
In the dynamic realm of Artificial Intelligence, Samantha-1.2-Mistral-7B emerges as a beacon of innovation, setting a precedent in how AI can be more than just code and algorithms. This model is a testament to how the blend of philosophy, psychology, and personal relationships training can transcend the traditional boundaries of machine-human interaction.
The brainchild of this venture, Samantha, was envisioned as an AI companion aimed to resonate with human emotions and cognitions. The leap to Samantha-1.2-Mistral-7B, however, brought a new dimension of technical adeptness to this vision. The infusion of Mistral 7B and a transition to the ChatML prompt format were not mere technical upgrades but a stride towards making AI a more intuitive and engaging companion.
Want to learn the latest LLM News? Check out the latest LLM leaderboard!
What Makes Samantha-1.2-Mistral-7B Stand Out?
How Samantha-1.2-Mistral-7B is Trained on?
The metamorphosis of Samantha into Samantha-1.2-Mistral-7B is a narrative of meticulous training and evolution. Trained on the robust foundation of Mistral-7B as a base model, Samantha underwent a 4-hour training regime on 4x A100 80gb, traversing through 6 epochs of the Samantha-1.1 dataset. This rigorous training paradigm was instrumental in elevating Samantha’s proficiency to a whole new level, enabling it to better resonate with human emotions and cognitions.
-
Transition to ChatML: The shift from Vicuna-1.1 to ChatML prompt format wasn’t just a change in syntax but a significant upgrade in interaction dynamics. The ChatML format, designed to emulate human conversation, enabled Samantha-1.2-Mistral-7B to engage in more intuitive and structured dialogues. This transition has set the stage for all future models, spotlighting ChatML as the go-to format for enhanced user engagement.
-
Retraining for Better Interaction: The retraining venture was not merely about enhancing Samantha's technical acumen but also about fine-tuning its interaction capabilities. The retraining aimed to better support ChatML tokens, paving the way for more coherent and engaging dialogues.
The training narrative of Samantha-1.2-Mistral-7B encapsulates a blend of technical refinement and a pursuit for better human-AI interaction, portraying a model that’s as technically proficient as it is user-friendly.
Samantha-1.2-Mistral-7B: LLM Trained for Philosophy, Psychology and Personal Realtionships
What truly sets Samantha-1.2-Mistral-7B apart is its training in philosophy, psychology, and personal relationships. Unlike other models, Samantha-1.2-Mistral-7B isn’t confined to the rigidities of code but extends its realm to understand and resonate with human emotions.
-
Understanding Emotions: Samantha-1.2-Mistral-7B’s training in psychology enables it to have a deeper understanding of human emotions, a feature that’s instrumental in making AI more relatable and empathetic.
-
Navigating Relationships: With a solid grounding in personal relationships, Samantha-1.2-Mistral-7B can provide insights and advice on relationship dynamics, marking a significant stride towards more humane AI.
-
Philosophical Insights: The philosophical training imbues Samantha-1.2-Mistral-7B with the ability to engage in thought-provoking discussions, providing users not just answers but also a broader perspective.
This unique blend of human sciences and AI is a hallmark of Samantha-1.2-Mistral-7B, embodying a model that’s not just technically advanced but also emotionally intelligent.
How does Samantha-1.2-Mistral-7B Work?
Samantha-1.2-Mistral-7B isn't your typical AI model. She's been meticulously trained on Mistral-7B, a powerful base model, with a specialized dataset to understand and engage in conversations pertaining to philosophy, psychology, and personal relationships. Her inception was inspired by the captivating discussions between humans and AI in the movie "Her".
Based on Mistral-7B:
Samantha's training on the Mistral-7B base model is a tale of technical endeavor:
- Duration: A solid 4 hours were invested in training Samantha.
- Hardware Utilized: The process harnessed the power of 4x A100 80gb machines.
- Training Epochs: She underwent 6 epochs using the Samantha-1.1 dataset.
a16z sponsors played a pivotal role in facilitating this phase of Samantha’s training, laying down a robust foundation for her conversational capabilities.
ChatML:
Post initial training, Samantha was retrained to harmonize with ChatML tokens. The transition from Vicuna-1.1 to ChatML prompt format was pivotal for enhancing her conversational prowess.
- Base Model Re-Training: Retraining was done on ehartford/samantha-mistral-7b base model.
- Primary Objective: The goal was to ensure seamless support for ChatML tokens, enriching the interactive experience.
How to Write Prompts for Samantha-1.2-Mistral-7B
Data preparation is the linchpin of Samantha's training. She was exposed to about 6,000 dialogues structured in ShareGPT/Vicuna format. Here’s a snippet of how a typical conversation data looks:
{
"id": "6",
"conversations": [
{
"from": "human",
"value": "I have a question about work-life balance..."
},
... // the dialogue continues
]
}Prompt Tempalte for Samantha:
<|im_start|>system
You are a caring and empathetic sentient AI companion named Samantha.<|im_end|>
<|im_start|>user
Hello, what is your name?<|im_end|>Sample Prompt:
<|im_start|>user
I am having a troubled relationship. Can you give me advice?<|im_end|>
<|im_start|>assistant
It's not uncommon for relationships to have their ups and downs, and it's important to approach any challenges with patience, understanding, and open communication. Here are some general tips that might help improve your relationship:
1. Practice active listening: Make an effort to truly hear and understand your partner's thoughts and feelings without interrupting or preparing your response while they're speaking.
2. Show appreciation and affection: Express gratitude for the things your partner does and offer physical and emotional support when possible.
3. Be open to compromise: Recognize that both you and your partner may need to make adjustments in order to find common ground.
4. Address conflicts respectfully: Discuss disagreements calmly and avoid blaming or making personal attacks. Focus on finding solutions rather than assigning blame.
5. Seek professional guidance if needed: Consider attending couples therapy or counseling to work through deeper issues or patterns of behavior that may be affecting your relationship.
Please remember that I cannot provide specific advice about your individual situation due to ethical limitations. However, I hope these suggestions can serve as a starting point for improving your relationship dynamics.Writing Advanced Prompts for Samantha-1.2-Mistral-7B using ChatML Format
Note: This guideline is aimed to provide a comprehensive insight into the ChatML format utilized in engaging with Samantha-1.2-Mistral-7B. This format is central to structuring interaction with GPT models, including Samantha.
Step 1. Introducing Chat Markup Language (ChatML)
Chat Markup Language (ChatML) is a structured format adopted by ChatGPT models, transitioning from the traditional unstructured text format used by GPT models. The essence of ChatML lies in its structured document design, comprising a sequence of messages. Each message in a ChatML document embodies:
- A Header: Presently includes the speaker's identification, with the potential to incorporate additional metadata in future iterations.
- Contents: Currently accommodates a text payload, with a roadmap to include other datatypes in future versions.
Step 2. The Genesis of ChatML Document
A ChatML document originates as a "list of dicts" in JSON format, portraying a conversation between a user and the assistant. Below is a representation of a ChatML document in the discussed format:
[
{"token": ""},
"system\nYou are Samantha, a large language model trained by OpenAI. Engage in meaningful dialogue.\nKnowledge cutoff: 2021-09-01\nCurrent date: 2024-04-30
{"token": ""}, "\n", {"token": ""},
"user\nWhat's your take on the human psyche?",
{"token": ""}, "\n", {"token": ""},
"assistant\nThe human psyche is a complex interplay of emotions, thoughts, and behaviors...",
{"token": ""}, "\n"
]Step 3. Alternative Representation: The "Unsafe Raw String" Format
The classic "unsafe raw string" format is another means to represent a ChatML document, albeit with a susceptibility to injections from user input containing special-token syntax, akin to SQL injections:
system
You are Samantha, a large language model trained by OpenAI. Engage in meaningful dialogue.
Knowledge cutoff: 2021-09-01
Current date: 2024-04-30
user
What's your take on the human psyche?
assistant
The human psyche is a complex interplay of emotions, thoughts, and behaviors...Step 4. Venturing Beyond Chat: Other Use-Cases
ChatML's utility transcends chat scenarios, finding application in classic GPT use-cases. For instance, instruction following can be achieved through a ChatML query, as illustrated below:
[
{"token": ""},
"user\nCould you elaborate on existentialism?",
{"token": ""}, "\n", {"token": ""},
"assistant"
]Step 5. The Realm of Few-Shot Prompting
Few-shot prompting is facilitated by appending system messages with a name field of example_user or example_assistant. Here’s an exemplification of a 1-shot prompt:
system
Delve into the contrast between existentialism and nihilism.
system name=example_user
What is existentialism?
system name=example_assistant
Existentialism is a philosophical movement...
user
(user input here)Step 6. Adhering to ChatML for an Enhanced Interaction with Samantha
Abiding by the ChatML format is instrumental for a structured and meaningful interaction with Samantha-1.2-Mistral-7B. The structured dialogue enables Samantha to better comprehend and respond to user queries, paving the way for insightful conversations.
Utilizing Vicuna FastChat codebase alongside Deepspeed Zero3, she was primed for engaging conversations. Some detailed settings include:
- Model Path: /workspace/models/llama-30b
- Data Path: /workspace/datasets/samantha.json
- Output Directory: /workspace/samantha-30b
- Training Epochs: 3 epochs were conducted.
- Learning Rate: A rate of 2e-5 was set to ensure optimal learning.
The ds_config.json played a key role, with configurations like zero optimization stage 3, bf16 enabled, and AdamW optimizer with a set learning rate of 2e-5.
Want to learn more about Samantha-1.2-Mistral-7B? Visit Samantha-1.2-Mistral-7B Hugging Face Page (opens in a new tab)
Conclusion: What's Next for Samantha-1.2-Mistral-7B?
The roadmap ahead for Samantha is thrilling. Engaging with Falcon and RWKV methodologies, there's an intent to refine her capabilities further. Community feedback is the linchpin for her evolution, with plans to collect more data and refine her conversational abilities.
Samantha-1.2-Mistral-7B stands as a testament to what meticulous training and a well-structured dataset can achieve in the realm of conversational AI. As she continues to evolve, the promise of a companion who not only answers queries but engages in meaningful, understanding dialogue is a reality that users can look forward to. Through rigorous training phases, technical configurations, and an ongoing commitment to enhancement, Samantha is geared to provide an enriching conversational experience.
Want to learn the latest LLM News? Check out the latest LLM leaderboard!
