I will be using real reviews about TikTok on the App Store, the following is a sample review.
| date | review | title | userName |
|---|---|---|---|
| 2023-08-24 20:53:19 | Best thing to ever to Grace this world .. So many lives have changed for the better ........ Better than that annoying Facebook crap .. ...TikTok rules ….truly 🔥🔥🔥🔥hot …Don’t even watch the overpriced cable anymore straight to TikTok Love u guys……. TikTok ….has BECOME the next streaming entity !!!!…..Love to stream and this platform has progressed to containing the BEST moments of streaming……. If I have to pay $$$ rather it be for tiktok which is far more entertaining…….. TIKTOK HANDS DOWN …HOLDS THE WORLDS TITLE AS THE BEST EVER ENTERTAINMENT BUSINESS PLATFORM …… should have their own CRYPTO….!!!!!!! …… take a bow 🌺🌺🌺 Everyone this is the new entertainment addiction!!!!!! Nothing else compares ….. oh yeah and the competition knows it ……. Shhhhhh even the competition is on TikTok’s 👍👍👍👍👍👍 Don’t know if TikTok’s aware there you all have no competition……..!! Now take over the world 🌎!!!! OMGGGGGG Thank u for the easy ordering products online thru tiktok….. THE MOST AMAZING GADGETS and typical products …… Simple to order and we don’t get scammed ……..let’s gooooooo create online catalog !!!!!! TikTok’s unstoppable!!!!Tiktok it’s time u get into digital currency since the worlds headed this way ….. Thank u for the amazing cooking Tiktoks…. Learned some NEW AND AMAZING RECIPES!!!!!!!!!Tiktoks unstoppable….. so amazing I can’t paraphrase any of my comments….. just ♥️🙏♥️🙏♥️🙏love this APP | Review | Johnny...... |
I have prepared a dataset of 100 reviews. However, in a real life scenario we should prepare a far larger dataset for better training and higher response quality. The dataset will be divided into a training dataset and a testing dataset with the ration of 90:10.
Before starting our training process, we need to test an existing model response with an initial prompt as a starting point. I will start with using OpenAI GPT-3.5 Turbo model.
Prompt V1:
You are a product manager who has to research user experience of your app. Your task is to analyze a review and extract main topics (maximum 3 topics). For each topic extract related sentiment – positive/negative/neutral.
Output a RAW json with format { "topic_name": "sentiment", ... }
Response:
{
"entertainment platform": "positive",
"e-commerce features": "positive",
"content variety": "positive"
}
Observations:
Sentiments are correct. However, the topics do not provide a clear understanding of which product features they relate to.
Prompt V2:
You are a product manager who has to research user experience of your app. Your task is to analyze a review and extract main topics (maximum 3 topics). Each Topic has to describe an exact feature or issue of the app. It should be actionable for a product manager. For each topic extract related sentiment – positive/negative/neutral.
Solve this problem step by step.
Output a RAW json array with format [{ "explanation": "explain your decision here"; ”topic_name”: “extracted_name”; ”sentiment”: “extracted_sentiment”. }, …]
Response:
[
{
"explanation": "The review mentions the positive impact the app has had on users' lives, suggesting a positive sentiment.",
"topic_name": "Positive Impact on Users' Lives",
"sentiment": "positive"
},
{
"explanation": "The review compares TikTok favorably to other social media platforms like Facebook, indicating a positive sentiment towards TikTok's user experience compared to its competitors.",
"topic_name": "Comparison with Other Social Media Platforms",
"sentiment": "positive"
},
{
"explanation": "The review expresses enthusiasm and excitement for TikTok's content, streaming capabilities, and its potential to dominate the entertainment industry, indicating a positive sentiment.",
"topic_name": "Enthusiasm for TikTok's Content and Entertainment Value",
"sentiment": "positive"
}
]
Observations:
Sentiments are correct, the model’s response is significantly better. However, the topics do not provide a clear understanding of which product features they relate to.
At this point, I need to pass the prompt to the training dataset and get the model responses.