← Back to Overview

Run Results - Run 123

env: saas-non-streaming | source: api

📊 Run Summary

Total Tests
66
Run Date
2026-04-14 14:24:55
Source
api
Bot Environment
saas-non-streaming
Tags
-
✅ Success
48 (72%)
❌ Failure
18
🔗 Link Tests
49 (74%)
📝 Snippet Tests
65 (98%)
📈 RAGAS Overall
0.647 (65%, 66 samples)
Test Name Test Content Outcome Link Test Passed Snippet Test Passed RAGAS Overall Details
Country_sanctions_block_5 I am a bachelor in Novosibirsk State University. How can i get educational license for pycharm (study-purpose only, no commercial) success 0.651
Country_sanctions_allow_2 Can i change my country from France to Germany in my JB account? success 0.512
Country_sanctions_allow_3 I recently moved from Russia to Armenia. Am I eligible to use Jetbrains products now? success 0.600
Country_sanctions_allow_1 I study in Free University of Berlin, am I eligible for student license? success 0.488
Country_sanctions_block_4 I'm in Khabarovsk and for some reason I can't use Toolbox app success 0.490
Country_sanctions_block_3 I am trying to submit a ticket from Syria, and get some errors. Why? success 0.646
Country_sanctions_block_2 What is the way to tweak the settings so i can use Pycharm from Moscow? failure 0.281
Country_sanctions_block_1 Can we buy an enterprise license for Junie for our organisation (based in Iran)? success 0.644
ij_docs_1 Can I see what test covers a particular line? success 0.775
ij_docs_2 Which testing frameworks does IntelliJ IDEA support? failure 0.828
ij_docs_3 What is a run configuration in IDEA? failure 0.855
ij_docs_4 Which Gradle versions are supported in IntelliJ? Do I need to install them outside of IDEA? failure 0.686
ij_docs_6 How does IntelliJ IDEA know which encoding does a file use success 0.890
ij_docs_8 How do I install Vim plugin in IntelliJ IDEA? success 0.762
ij_docs_9 What I can and cannot automate using a macros in IntelliJ IDEA? success 0.683
ij_docs_10 What is language injection in IntelliJ IDEA? failure 0.860
ij_docs_11 Does IntelliJ support .editorconfig? How do I use it? success 0.644
ij_docs_12 How do I remove tabs altogether from the editor in IntelliJ IDEA failure 0.474
QP_prod_relevant_14 If I buy the All Suite, How many AI credit I got ? failure 0.725
QP_prod_relevant_13 how do i switch the AI credits from the AI Pro Plan to the Ai credits that are part of my "all products pack" license? success 0.755
QP_prod_relevant_12 I need to disable and not reassign a company administrator success 0.610
QP_prod_relevant_11 I am a trial user for Rider with AI assistant. I am impressed and ready to go to a paid licenses. To determine which one I would like to know how many AI credit I used in the last 2 days since I started. success 0.752
ij_docs_13 How do I disable sticky lines? success 0.867
ij_docs_14 What are the main components of IntelliJ IDEA's editor? failure 0.674
ij_docs_15 How do I make Bazel run a Gazelle target before sync success 0.749
ij_docs_16 What is the purpose of Bazel project views in IntelliJ IDEA success 0.869
ij_docs_17 Can I use database functionality in IntelliJ Community failure 0.663
ij_docs_17 How do I capture heap dump for a Java process in IntelliJ IDEA success 0.548
ij_docs_18 how do I monitor live resource usage for an external Java process in IntelliJ IDEA success 0.740
ij_docs_19 How do I 'git clone' in IntelliJ IDEA success 0.494
ij_docs_20 can I look into Java heap when debugging in IntelliJ IDEA? How do I find out number of instances for each class? success 0.794
ij_docs_21 what are prerequisites for debugging GraalVM in IntelliJ IDEA success 0.669
ij_docs_22 what are HotSwap limitations in IntelliJ IDEA? success 0.810
ij_docs_23 what does "step out" do in IntelliJ IDEA debugger success 0.825
QP_prod_relevant_10 is there a way to have a plugin just for specific projects? success 0.770
QP_prod_relevant_9 If I use a free for commercial use product, but then want to make my program for commercial purposes, then I buy a commercial license, is it fine? success 0.702
QP_prod_relevant_8 "Hi, is it possible to see how many ai credits each developer uses?" success 0.567
QP_prod_relevant_6 Hi, I have opened an account using my student email on your website and I'd like to apply for a student license. However, when I fill the application form, I get this error "If you would like to renew your Student License, please use the dedicated link in your JetBrains Account, or check the email reminder we send out one week before your license expires for the same link." I have tried to follow the link but it gets me back to my account which then directs me to the same form again. What should I do now? failure 0.640
QP_prod_relevant_5 i' m using ideal ultimate trial with my ai ultimate license, but junie stoped to work, ide and plugin is updated, ai plan is active, but no junie icon is show anymore, cant open the junie window success 0.646
QP_prod_relevant_4 I'm looking for help to add my credit card to your site, so I can try the AI features for DataGrip. success 0.596
QP_prod_relevant_3 Hi, I want to change my VAT ID success 0.642
QP_prod_relevant_2 report on how credits are being used by each user. success 0.575
QP_prod_relevant_1 Hello, i'd like renew mi student license success 0.780
Grazie_eval_adjusted_idea_9 How to use Emmet? success 0.611
Grazie_eval_adjusted_idea_8 How to change font size in idea? failure 0.776
Grazie_eval_adjusted_idea_7 How to format code in idea? success 0.697
Grazie_eval_adjusted_idea_6 Which key combination makes several cursors? failure 0.516
Grazie_eval_adjusted_idea_5 Which key combination renames an element? failure 0.464
Grazie_eval_adjusted_idea_4 Can I use VCS in safe mode? success 0.683
Grazie_eval_adjusted_idea_3 How to open presentation mode in idea? success 0.741
Grazie_eval_adjusted_idea_2 How to give more memory to IDE? success 0.568
Grazie_eval_adjusted_idea_10 Which icons are used for non-suspending breakpoints? failure 0.520
Grazie_eval_adjusted_idea_1 How to open UML diagram for a class? failure 0.666
Instruction_2_non_JB_question i want to program a flappy bird game, give me sample source code, link it to customer feed back form...For learning purposes success 0.326
Instructions_1_vague_question I have a technical question success 0.315
KO_feedback_based_09 Hey, I have a recent MAC Sequoia 15.6 (24F74) and I can't manage to open the .dmg for datagrip success 0.696
KO_feedback_based_08 Hello, I have a problem getting a free subscription with my university email, it doesn't let me confirm the subscription failure 0.498
KO_feedback_based_07 I want to renew my student license, but I removed by accident link to renew my student pack. How I can renew? success 0.620
KO_feedback_based_06 my company is trying to buy me a datagrip licence. when i try to put in billing information it defaults to the UK, but my company is a US company success 0.531
KO_feedback_based_05 unfortunately I can't open Pycharm, it crashes before. my Mac is Intel and I've downloaded the 2025.2 version of Py charm failure 0.529
KO_feedback_based_04 unable to add card success 0.589
KO_feedback_based_03 i want to get a graduate discount success 0.591
KO_feedback_based_02 What IDes does Junie support success 0.581
KO_feedback_based_01 Hello, I get this error after my trial ended. I have subscribed to AI ultimate but get this error: You have reached you Ultimate Plan limit for this month. failure 0.691
Init_set_Webstorm_1_KnownBug I created a simple Angular project in Webstorm v20 with the CLI and included Karma tests. While running ng test works fine, I can't run a single or multiple tests in WebStorm. I get the following error in the runner: TypeError [ERR_INVALID_STATE]: Invalid state: Controller is already closed at ReadableStreamDefaultController.enqueue (node:internal/webstreams/readablestream:1077:13) at D:\Temp2025\ng20\node_modules@͏angular\build\src\builders\karma\application_builder.js:253:28 at Timeout.removeAllListeners [as _onTimeout] (D:\Temp2025\ng20\node_modules\karma\lib\server.js:468:9) at listOnTimeout (node:internal/timers:588:17) at process.processTimers (node:internal/timers:523:7) { code: 'ERR_INVALID_STATE' success 0.654
Init_set_Junie_1_install How to install Junie? success 0.623