Overview
Rust has emerged as a prominent system programming language, distinguished by its robust memory and type safety features, without sacrificing performance. This positions Rust as a compelling alternative to conventional languages such as C and C++, which are known for their susceptibility to memory security vulnerabilities. Manual conversion of C code to Rust, however, demands significant effort from developers due requiring in-depth knowledge of Rust's language features.
To mitigate this effort, automated C-to-Rust refactoring tools have been developed. These tools aim to generate safe Rust code through the application of static analysis and Large Language Models (LLMs). While proponents of these tools suggest they enhance safety by reducing the prevalence of unsafe Rust code, the direct correlation between this reduction and improved memory security remains undefined.
Research Context
The research addresses the ambiguity surrounding the effectiveness of automatic C-to-Rust refactoring tools in delivering actual memory security benefits. Despite claims of enhanced safety through the reduction of unsafe Rust code, a clear link to broader security improvements has not been established. This study seeks to provide empirical data on the reliability, safety, and correctness of various C-to-Rust refactoring methodologies to clarify this relationship.
Approach
The study conducted a comprehensive empirical evaluation of four specific C-to-Rust refactoring methods: C2Rust-analyze, CROWN, C2SaferRust, and FLOURINE. The evaluation utilized a dataset comprising 116 C programs, each containing identified memory security bugs, sourced from the NIST Juliet Test Suite.
From these 116 C programs, the tools generated a total of 464 Rust programs. The evaluation focused on three primary aspects:
- Compilation Correctness: Assessing whether the refactored Rust programs successfully compiled.
- Effectiveness in Mitigating Original C Bugs: Determining if the refactored Rust programs successfully addressed and mitigated the memory security bugs present in the original C code.
- Introduction of Additional Rust Bugs: Identifying whether the refactoring process inadvertently introduced new memory security vulnerabilities specific to Rust.
The research also involved an examination of the design rationale underpinning each tool and an analysis of the root causes of errors observed across the different refactoring methods.
Findings
The empirical study yielded several key findings regarding the performance and effectiveness of automatic C-to-Rust refactoring tools:
- Compilation Failures: A substantial number of generated Rust programs failed to compile. Specifically, 342 out of the 464 Rust programs generated by the tools were unable to compile.
- Inherited Memory Security Bugs: The refactoring process frequently failed to eliminate existing vulnerabilities. 177 of the generated Rust programs retained memory security bugs from their original C counterparts.
- Introduction of New Bugs: The tools also introduced new vulnerabilities into the converted code. 77 new Rust-specific bugs were observed in the generated programs.
An analysis of tool design rationales and error root causes indicated that current automated refactoring tools fulfill their internal definition of memory safety. However, they do not consistently deliver the broader objective of memory security when adopted in practice.
Why This Matters
The findings from this study suggest that the current generation of automated C-to-Rust refactoring tools, while aiming to enhance memory safety, do not consistently achieve the broader goal of memory security. The observed rates of compilation failures, inherited bugs, and newly introduced vulnerabilities highlight a gap between the intended function and practical outcomes. This implies that relying solely on these tools may not fully address the security concerns associated with C and C++ codebases when migrating to Rust.