Prompt: Drug Review

Settings

Model:
gpt-4o-mini
Metric:
Exact Match
Dataset:
Drug Reviews
Dataset size:
100

Performance

Training step(slide to see different versions of prompts)

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12

System prompt

Task description: Drug Review Rating.

Original

Optimized (step 1): 
Rate the drug from 1 to 10 based on the following criteria: effectiveness (40%), side effects (30%), and overall user satisfaction (30%). Carefully read the review and consider the context, including the severity of the condition being treated and the user's overall experience. Pay special attention to both positive and negative aspects mentioned, especially those that significantly impact the user's quality of life. If the review contains mixed sentiments, assign a preliminary score for each criterion and then average these scores, adjusting for the severity of any negative aspects. For longer reviews, summarize the key points related to effectiveness, side effects, and user satisfaction by identifying statements that directly mention changes in symptoms, quality of life, or specific side effects before assigning a rating. Ensure that similar reviews—those with comparable conditions, drug names, and overall sentiments—receive consistent ratings, even if phrased differently. Only output a single number from 1 to 10, with no additional text or punctuation.

User prompt

Description: Information about the drug and its review.

Prompt for Input Variables:
Review: $review
Condition: $condition
Drug Name: $drug_name

Few Shot Examples

Note: The few-shot examples work together with the optimized system prompt. Please include in the final prompt for best results.
EXAMPLE 1:

Review: "Had experienced ED for awhile due to antidepressants and other anxiety meds, so needed a little extra boost to get my confidence back. Got a free trial of Cialis and it was great! Started out taking 5mg daily. Within about 6 hours of taking the first dose, I suddenly got turned on and wow it really kicked in! Gave me rock hard erections. I felt a bit flushed that first day or two. Then the moment arose and I had sex on it, and wow. I was like an 18 yo again. I didn't have to worry about things not working, I could just focus on enjoying, which was great! It was great because I got rock hard anytime I was turned on. I did notice I felt very flushed and also had bad lower back and leg pain the first week or so. But that went away. I loved it!"
Condition: erectile dysfunction
Drug Name: cialis
Ground Truth Answer: 9

EXAMPLE 2:

Review: "This makes month 3 for me on Aviane and so far so good! I'm taking this to prevent pregnancy and to regulate my period. I had taken Yaz before but then had some side effects so I switched. For the most part, my period is consistent."
Condition: birth control
Drug Name: ethinyl estradiol / levonorgestrel
Ground Truth Answer: 8