Abstract:  Large language models (LLMs) like GPT-4 have shown potential in generating scientific feedback on research papers, but their effectiveness is limited by vagueness, a lack of domain-specific insights, and insufficient technical critiques, especially regarding model architecture and design. This study aims to address these limitations and enhance the LLM-based feedback system for research papers. The identified gaps include the need for specific, actionable feedback and domain-specific expertise. Our multi-faceted approach includes fine-tuning LLMs with domain specific datasets, incorporating expert-driven feedback, and focusing on detailed, section-specific comments. We also introduce specificity metrics, hybrid models combining LLM and human reviews, and iterative feedback mechanisms. These strategies aim to enhance the quality and utility of LLM-generated feedback, making it more actionable and aligned with expert human reviews. The proposed improvements could significantly reduce the number of review cycles before publication, providing timely and relevant feedback to authors. This research fills critical gaps in existing feedback systems, offering a robust solution to improve the academic review process.

Index Terms: Large Language Models (LLMs), GPT-4, Scientific Feedback, Domain-Specific Datasets, Expert-Driven Feedback, Section-Specific Comments, Specificity Metrics, Hybrid Models, Academic Review Process.

| DOI: 10.17148/IARJSET.2024.111208