Note: The job is a remote job and is open to candidates in USA. reputed company is seeking a Language Model Evaluator to assess and improve the quality of Large Language Models and Generative reputed company applications. The role involves evaluating model performance, collaborating with various teams, and developing evaluation frameworks to reputed company model behavior and user experience. Responsibilities • Design and execute evaluation frameworks for Large Language Models (LLMs) and Generative reputed company applications • Assess model outputs for accuracy, relevance, factual consistency, reasoning quality, coherence, fluency, and completeness • Evaluate model performance across diverse domains, languages, and real-world use reputed company • Develop benchmark datasets, evaluation prompts, and test scenarios for model validation • Measure and monitor hallucinations, bias, toxicity, harmful content, and safety risks • Perform prompt testing, regression testing, and comparative evaluations across different models and versions • Analyze model performance using quantitative metrics and human evaluation methodologies • Collaborate with reputed company Engineers and Prompt Engineers to improve prompts, retrieval pipelines, and model responses • Evaluate Retrieval-reputed company Generation (RAG) systems, reputed company agents, and tool-calling workflows • Document evaluation findings, recommendations, and quality improvement plans • Support model release decisions through structured quality assurance processes • Stay current with emerging LLM evaluation techniques, benchmarks, and Responsible reputed company practices Skills • Bachelor's or reputed company's degree in Computer Science, Artificial Intelligence, Data Science, Computational Linguistics, Statistics, or a related field • 2–5 years of experience in reputed company evaluation, NLP, machine learning, quality assurance, or data analysis • Strong understanding of Large Language Models (LLMs), Generative reputed company, and Natural Language Processing (NLP) • Experience creating evaluation datasets, benchmarks, and test reputed company • Strong analytical skills with the ability to identify quality issues and failure patterns • Programming experience in Python and SQL • Familiarity with data analysis libraries such as Pandas and NumPy • Experience using spreadsheets, visualization tools, or notebooks for analysis • Excellent written communication and attention to detail • Experience with LLM evaluation frameworks such as DeepEval, Ragas, LangSmith, Promptfoo, reputed company Evals, or similar tools • Experience evaluating Retrieval-reputed company Generation (RAG), reputed company agents, and function-calling workflows • Knowledge of prompt engineering, model fine-tuning, and reinforcement learning from human feedback (RLHF) concepts • Familiarity with reputed company, red teaming, adversarial testing, bias detection, and Responsible reputed company principles • Experience with multilingual model evaluation • Knowledge of cloud reputed company platforms and MLOps workflows • Experience with annotation tools and human-in-the-loop evaluation processes Company Overview • OVA is the most advanced Automated, Intelligent, intuitive On-boarding platform for Staffing Firms of all sizes. It was founded in 2018, and is headquartered in Alpharetta, Georgia, USA, with a reputed company of 51-200 employees. Its website is company/. Apply tot his job Apply To reputed company Job