Research Programmer

VK is looking for a Research Programmer to participate in designing RAG architecture for B2B cases and work with LLMs. The role involves data ingestion, evaluation pipelines, and integration with infrastructure. VK is a leading technology company that develops a wide range of products and services, focusing on social media, communication, and entertainment.

• •Participate in designing RAG architecture for B2B cases: help choose data sources, set up ingestion, chunking, basic retrieval strategies (dense/hybrid) under the guidance of more experienced colleagues;
• •Configure LLM/agents for the subject area: implement prompts, connect tools (tool calling), work with context and basic memory;
• •Implement POC/POV on client data, conduct iterative improvements based on metrics and feedback;
• •Build Data Ingestion pipelines with OCR;
• •Implement evaluation pipelines: generate test sets, run auto-eval, participate in human-eval;
• •Calculate and analyze key metrics (accuracy, coverage, latency, user success rate), prepare reports for the team;
• •Participate in setting up monitoring: logging, basic alerts, analyze unsuccessful cases and their fixes;
• •Participate in integration with infrastructure (Kubernetes, cloud), work with metrics and logs;
• •Participate in load testing to assess inference speed and necessary resources.

• •Experience with Python (pandas, PyTorch/TF, API LLM);
• •Experience in ML/DS/NLP for at least two years;
• •Experience with LLM/RAG;
• •Understanding of the full ML development cycle;
• •Practical experience with RAG: setting up retrieval, working with embedding models, vector databases (FAISS, Qdrant, Pinecone, etc.);
• •Understanding of basic quality improvement techniques (reranking, filtering, prompt tuning);
• •Experience with agent frameworks (LangChain / LangGraph / LlamaIndex / ADK, etc.);
• •Basic understanding of agentic approaches and tool calling;
• •Experience in prompt engineering and configuring LLM for tasks;
• •Basic experience in fine-tuning (or understanding approaches: LoRA, QLora, instruction tuning);
• •Understanding of inference optimization (at the level of using tools: vLLM, quantization, etc.);
• •Experience with popular ML frameworks: transformers, vllm, sglang, pytorch;
• •SQL, basic understanding of working with data and pipelines;
• •Experience deploying services (Docker, FastAPI), basic understanding of Kubernetes/cloud;
• •Skills in logging, monitoring, and basic A/B experiments;
• •Experience with evaluation for ML/LLM (auto-metrics + participation in manual evaluation, LLM-as-Judge).