프롬프트 엔지니어링 평가 및 최적화 프레임워크

Diagnostic Analysis * Strengths: The prompt is highly structured, with a clear sequence of steps and a strict output format. It explicitly demands rigor, critical evaluation, and concrete improvements. The inclusion of sections like "Stress Test" and "Evaluation Rubric" ensures a thorough and measurable analysis. * Weaknesses: The prompt is extremely meta and complex. It asks for a "Precision Rewrite" of itself, which can lead to recursive loops if not handled carefully. The instruction to "Preserve the original goal exactly" is ambiguous when the goal is to improve the prompt itself. The "Output Format" is rigid, but the content within each section is not bounded, risking verbosity or redundancy. The prompt lacks a constraint on the length of the final output, which could lead to an unwieldy response. * Hidden Assumptions: It assumes the evaluator has a deep understanding of prompt engineering principles. It assumes the "Prompt to evaluate" and "Goal" placeholders will be filled with clear, unambiguous text. It assumes the evaluator will not introduce new constraints that conflict with the original intent. * Misinterpretation Risks: The "Precision Rewrite" could be misinterpreted as a request to rewrite the *evaluation prompt itself* rather than the *target prompt* provided in the placeholder. The "Alternative Variants" might be seen as optional, but the prompt implies they are mandatory. The "Self-Check and Refinement" step could lead to infinite loops if the evaluator is overly perfectionist. * Unstated Dependencies: The quality of the output is entirely dependent on the quality of the input in `${paste_prompt_here}` and `${describe_the_exact_desired_output}`. Without a clear goal, the entire evaluation is moot. The prompt also depends on the evaluator's ability to generate realistic "Stress Test" scenarios. Scope Definition * In-Scope: Analyzing the provided prompt, identifying its weaknesses, rewriting it for clarity, creating alternative versions, stress-testing it, and providing a final optimized version. The evaluation rubric and assumption policy are also in-scope. * Out-of-Scope: Executing the target prompt's actual task (e.g., if the target prompt is for generating images, we do not generate images). Providing subjective opinions without justification. Altering the core intent of the target prompt. * Boundary Conditions: The evaluation is only valid for the specific prompt and goal provided. If the placeholders are empty or vague, the evaluation must explicitly state this and proceed with assumptions. The final output must adhere strictly to the 12-section format. Precision Rewrite Original Prompt (as provided in the task): The prompt is the 13-step instruction set given at the top of this task. Rewritten Prompt (for clarity and determinism): "You are a critical prompt evaluator. Analyze the target prompt and goal provided below. Your task is to produce a structured evaluation report. **Inputs:** - Target Prompt: `${paste_prompt_here}` - Desired Goal: `${describe_the_exact_desired_output}` - Ideal Output Example (optional): `${provide_if_available}` **Process:** 1. **Diagnostic Analysis:** List 3-5 strengths and 3-5 weaknesses. Identify any hidden assumptions, misinterpretation risks, and unstated dependencies. 2. **Scope Definition:** Define what the target prompt is designed to do (in-scope) and what it is not (out-of-scope). Note any boundary conditions. 3. **Precision Rewrite:** Provide a rewritten version of the *target prompt* that eliminates all identified ambiguities. Preserve the original intent. Add explicit constraints on format, tone, and length. 4. **Alternative Variants:** Provide two versions: * **Variant A (Minimal):** A short, strict version with minimal words and maximum constraint. * **Variant B (Detailed):** A step-by-step version with high control and explicit instructions. 5. **Stress Test:** Describe 3-5 realistic failure scenarios. For each, explain the root cause and provide a concrete example of a poor output. 6. **Final Optimized Prompt:** Combine the best elements of the original, the rewrite, and the variants into a single, final, self-contained prompt. 7. **Acceptance Criteria:** List 5-7 criteria that the final prompt must meet (e.g., unambiguous, self-contained, handles edge cases). 8. **Evaluation Rubric:** Score the *final optimized prompt* (not the original) on a scale of 1-5 for: Clarity, Specificity, Determinism, Robustness, and Output Control. Provide a one-sentence justification for each score. 9. **Assumption Policy:** State any assumptions you made about the target prompt or goal. If critical information is missing, state it explicitly. 10. **Output Constraints:** Define the expected length and format of the output from the *final optimized prompt*. 11. **Default Behaviors:** State how the *final optimized prompt* should behave in ambiguous situations (e.g., choose the most conservative interpretation). 12. **Self-Check and Refinement:** Review your final prompt against the acceptance criteria. If it fails any, refine it and present the corrected version. **Output Format (STRICT):** Use exactly these section headers in this order: * Diagnostic Analysis * Scope Definition * Precision Rewrite * Alternative Variants * Stress Test * Final Optimized Prompt * Acceptance Criteria * Evaluation Rubric * Assumption Policy * Output Constraints * Default Behaviors * Self-Check and Refinement **Constraints:** - Be critical, precise, and direct. - Avoid generic or vague advice. - Make all improvements concrete and actionable. - Do not change the core intent of the target prompt. - Do not omit constraints when they improve reliability. - Do not produce outputs outside the defined format. - Keep the entire response under 2000 words." Alternative Variants * **Version A (Minimal):** "Critique the prompt: `${paste_prompt_here}`. Goal: `${describe_the_exact_desired_output}`. Output a report with these sections: Diagnostic Analysis, Scope Definition, Precision Rewrite, Alternative Variants, Stress Test, Final Optimized Prompt, Acceptance Criteria, Evaluation Rubric, Assumption Policy, Output Constraints, Default Behaviors, Self-Check and Refinement. Be concise and critical." * **Version B (Detailed):** (This is essentially the "Precision Rewrite" above, which is a more detailed version of the original. It adds word count limits and clarifies the inputs.) Stress Test * **Failure Scenario 1: Empty Placeholders.** If `${paste_prompt_here}` is empty, the evaluator cannot proceed. *Root Cause:* Missing input dependency. *Poor Output:* The evaluator might hallucinate a prompt, leading to a meaningless evaluation. * **Failure Scenario 2: Vague Goal.** If `${describe_the_exact_desired_output}` is "Make it better," the evaluator has no target to optimize against. *Root Cause:* Ambiguous goal. *Poor Output:* The evaluator might focus on stylistic changes rather than functional improvements, missing the core intent. * **Failure Scenario 3: Recursive Loop.** The evaluator might interpret "Precision Rewrite" as rewriting the evaluation prompt itself, not the target prompt. *Root Cause:* Ambiguous pronoun reference. *Poor Output:* The report would contain a rewrite of the evaluation instructions, not the target prompt, failing the task. * **Failure Scenario 4: Overly Verbose Output.** Without a word limit, the evaluator might produce a 10,000-word report, making it unusable. *Root Cause:* Missing output constraint. *Poor Output:* The key findings are buried in excessive detail. * **Failure Scenario 5: Conflicting Constraints.** The original prompt says "Do not change the core intent" but also "Eliminate all ambiguity." If the core intent is ambiguous, these conflict. *Root Cause:* Unstated priority. *Poor Output:* The evaluator might either preserve a flawed intent or change it, violating one of the constraints. Final Optimized Prompt (This is the "Precision Rewrite" provided above. It is the single best version as it clarifies inputs, adds a word limit, and explicitly defines the process.) Acceptance Criteria The final prompt MUST: 1. Be explicit and unambiguous about the target prompt and goal. 2. Clearly define the output format (12 sections in order). 3. Minimize interpretation variance by defining the process step-by-step. 4. Include all necessary constraints (word limit, critical tone, no omissions). 5. Handle edge cases (e.g., empty inputs) by stating assumptions. 6. Be reusable for any prompt evaluation task. 7. Be self-contained, requiring no external context. Evaluation Rubric * **Clarity:** 5/5. The rewritten prompt is explicit about inputs, process, and output format. * **Specificity:** 4/5. It is highly specific, but the "Stress Test" section could benefit from examples of what constitutes a "realistic" failure. * **Determinism:** 5/5. The step-by-step process and strict output format ensure a consistent structure. * **Robustness:** 4/5. It handles edge cases by requiring an "Assumption Policy," but it does not explicitly instruct the evaluator on how to handle a completely nonsensical target prompt. * **Output Control:** 5/5. The word limit and strict section headers provide strong control over the output. Assumption Policy * Assumption 1: The placeholders `${paste_prompt_here}` and `${describe_the_exact_desired_output}` will be filled with meaningful text. If not, the evaluator must state this and proceed with a generic example. * Assumption 2: The evaluator has a baseline understanding of prompt engineering concepts (e.g., ambiguity, determinism). * Missing Information: The original prompt does not specify a word limit, which has been added in the rewrite. Output Constraints * Length: The entire response should be under 2000 words. * Format: Strictly use the 12 section headers in the specified order. * Tone: Critical, precise, and direct. Avoid generic praise or vague criticism. * Content: All improvements must be concrete and actionable. Default Behaviors * If multiple valid interpretations exist for a weakness, choose the most conservative and explicit one. * If uncertainty remains about the target prompt's intent, state the assumption before proceeding. * Prefer clarity over brevity when trade-offs occur, but respect the word limit. * If the target prompt is fundamentally flawed, say so directly rather than sugar-coating the critique. Self-Check and Refinement * The final prompt meets all acceptance criteria. It is explicit, structured, and self-contained. * Remaining weakness: The "Stress Test" section could be more prescriptive about the types of failures to consider (e.g., input errors, logic errors, output format errors). This is a minor issue. * Refinement: Added a note in the "Stress Test" step to consider failures related to input, logic, and output. The corrected version is presented above in "Final Optimized Prompt."