OracleLLM is an open-source research organization dedicated to exploring and advancing the concept of LLMs-as-Oracles. LLMs-as-Oracle refers to replacing human-generated golden references with the output and feedback produced by large language models (LLMs), treating them as authoritative benchmarks and learning signals for various tasks and applications. Our research areas include: (1). Data synthesis and annotation with LLMs. (2). LLMs as evaluators or judges for various tasks. (3). Employing LLMs' self-feedback as guidance for planning, self-improving and decision-making. Check more details on our current progress including papers, projects, and reading list!
We sincerely thank the advisors and collaborators in our research work.