Archive

Archive for the ‘USMLE Scores’ Category

CS fail

November 2, 2011 Leave a comment

A question sent to me:
Will good US clinical experience and US letters overcome a CS fail? Do applications get screened for exam fails?

No and no.
CS fail is almost non existent in AMGs.
About 15% of IMGs fail the CS. An blog viewer sent me the likely reason for his CS fail was lack of prep time. He has a terrific score on step 1 and should have done well on the CS. He was working and only had a limited time to study and only a limited time off to travel to the US to take the exam. I think that those who fail the CS underestimate the difficulty of emulating the American exam style. Many people who do quite well on the steps fail the CS.

Remember the CS does not evaluate your clinical skills. It evaluates if your clinical skill style matches that of soon to be American grads. American grads don’t have trouble being understood and don’t write notes like an IMG. Most of the trouble with the CS is that it tests style. If you have not practiced in the correct style of the exam you will not do well no matter how accomplished you are in your own school or hospital. I know a lot of IMGs who fail the CS because of lack of preparation, not lack of knowledge. How else is it that an IMGs steps can be above 240 and fail the CS? It is not a knowledge issue it is a style issue. You must learn the American style of speaking to patients, and documenting your findings. Interestingly men do worse on the CS than women, see my other CS post for details.

The ERAS program allows for filtering of a lot of things. The ERAS program can filter for applications from a particular school, for a particular score on all exams or for US experience (but not specific experience like research, observerships or clerkship rotations as a student).

What the ERAS program can’t filter for is exam fails. So if you have scores above the filter cut offs for that particular program and have failed a step or a CS one can only exclude your application by looking at the examinations tab. The question is which candidate does a program take, the one with terrific step scores and a failed CS or the one with terrific scores and no failed CS?

Good candidates look very similar. A failed attempt at CS is hard to overcome. Can you overcome a fail? Sure and I know people who have but you are at a distinct disadvantage in the matching game.

For those of you who will take the CS, do not underestimate it.

The views expressed in this post are those of Kenneth Christopher, MD and do not necessarily reflect the views of Dr. Christopher’s employer Partners HealthCare.

What do Program Directors think of the Clinical Skills Exam?

The CS is an expensive exercise but in some ways an important one.

The Step 2 CS has three subcomponents, the Integrated Clinical Encounter (ICE), Communication and Interpersonal Skills (CIS), and Spoken English Proficiency (SEP).

Both scores from the Step 2 CS prototype significantly correlated with the interns’ quartile ranking and average competency score. See Academic Medicine:May 2005 – Volume 80 – Issue 5 – pp 496-501

There was little overlap between examinees failing Step 2 CK and the different components of Step 2 CS. See Academic Medicine:October 2006 – Volume 81 – Issue 10 – pp S21-S24

“The inclusion of a clinical skills examination in the testing system for medical licensure provides the opportunity to assess history-taking and physical examination skills, communication and interpersonal skills, spoken English proficiency, and documentation of findings in a structured patient note” J Med Licensure Discipline. 2005;91:21-25.

The following data is © 1996-2011 FSMB and NBME®.  First-taker passing rates on each of the CS subcomponents for AMGs and IMGs. For IMGs in 2009-2010 the scores were ICE is 85% for IMGs compared to 98% AMGs, CIS is 87% for IMGs compared to 99% AMGs, SEP is 95% for IMGs compared to >99% AMGs.

The following graph is data on gender differences in the CS exam (click figure to enlarge) from a study published in Academic Medicine:

This is from a sample of 24,000 AMGs and IMGs who took the CS exam.  Fifty-seven percent of the examinees were male, 45% were international graduates, and 42% reported having English as a second language so the data is representable for IMGs.  Looking at the curves, women do better than men in every category.  Candidates do poorly on their first encounter and gradually improve.  Note that there is a dip in performance after the fifth and ninth station, this is the encounter after the breaks.  English language ability was constant.

What can we learn about this that will help IMGs pass the CS?  Well if you are male you need to work on being nice and warm.  You need to get into the right frame of mind before you start, as it looks like everyone starts out “cold.”  Perhaps practicing your greeting over and over just before you begin can help like an actor warming up their vocal cords.  “Hi my name is Kenneth what brings you to the clinic.”  Remember to smile at your patients, be friendly and warm.  You should have practiced the structure of the note while studying.  I suggest writing a note while everyone is talking about nothing and waiting for the exam to begin.  Get your mind and pen  and words warmed up.  After the break again practice your greeting so that you are warmed up.  Don’t let your guard down and stay focused on documentation and data gathering.  Avoiding a poor start and decrease in performance after the breaks may be the difference between passing and not.  In essence the CS is part performance and part being prepared.

The views expressed in this post are those of Kenneth Christopher, MD and do not necessarily reflect the views of Dr. Christopher’s employer Partners HealthCare.

What are my chances?

Let me get out my crystal ball.

The “what are my chances” question is not one that can be answered as I do not have enough data.  One can predict the probability of matching based on Step 1 scores or a combination algorithm.

Remember one can not underestimate the importance of data.  According to the NRMP 2009 data the probability of matching in Internal Medicine (IM) is 35% if you have 3 interviews, 50% if you have 5 interviews, 75% if you have 10 interviews.  That makes sense you have to interview to match.  Many people are lucky to get 1 interview which corresponds to a 25% chance of matching in IM.

Regarding Step 1 scores, for those who do interview in IM, the probability of matching in IM with Step 1 200 is 27%, Step 1 of 220 is 45%, 240 is 63%.  Information on step 2 scores is not present as many AMGs don’t take step 2 prior to applying.

What can I do to improve my chances?  Just over half (1092/2113) of those who match do not have any Abstracts, Presentations, or Publications which suggests that if you are strong enough as a candidate, publications/research don’t matter.

The views expressed in this post are those of Kenneth Christopher, MD and do not necessarily reflect the views of Dr. Christopher’s employer Partners HealthCare.

2 digit USMLE scores are going away

August 4, 2011 1 comment

So the two digit score is going away, sort of.

From ECFMG Reporter issue 169 May 2011: “Starting July 1, 2011, USMLE transcripts reported through the ERAS reporting system will no longer include score results on the two-digit score scale. USMLE results will continue to be reported on the three-digit scale. This affects the Step 1, 2 CK, and 3 examinations only; Step 2 CS will continue to be reported as pass or fail. These changes do not alter the score required to pass or the difficulty of any of the USMLE Step examinations.”

This doesn’t have any effect on the applicant as most programs use the 3 digit scores in their filters anyway.  What it does mean for the IMG community is a frame shift in the thinking of what the scores mean.    What this means is that “99/99” will become a thing of the past.  No more “he is a 99/99 so he must be good.”  A 99 two digit score represented about the top 20% of scores, not the 99th percentile.  True 99th percentile is a three digit score of 260 according to a nice analysis by My Dominant Hemisphere. This was a big point of confusion for a lot of people for a long time.  The only people I ever heard mention the two digit score were IMGs.  And the mention bordered on obsession.  The two-digit scores were designed for the state medical boards and will still be reported to the states and to the test taker but not the residency programs.

On the two-digit scale, the minimum passing score is always a 75 but with the 3 digit score it depends on the distribution.  Minimum Passing three digit Scores for last year were Step 1 188, Step 2 CK 189, Step 3 187. My advice is to think of your scores in the three digit form and compare them to the NRMP data to see in which specialties that you might fit.

The views expressed in this post are those of Kenneth Christopher, MD and do not necessarily reflect the views of Dr. Christopher’s employer Partners HealthCare.

Categories: USMLE Scores Tags: , , , ,

Even excellent scores are no guarantee

August 2, 2011 2 comments

I previously looked at low USMLE scores and the likelihood of matching.

Lets look at the opposite.  What are the chances of getting a residency in IM with terrific scores?

Again lets look at the NRMP data from 2009.

In general, scores over 230 are considered very good.  Scores over 240 are terrific.

For Step 1, for the 1054 IMGs with scores above 230 who interviewed, only 61.7% matched to internal medicine.  For the 459 IMGs with scores above 240 who interviewed, only 70.6% matched to internal medicine.

For Step 2, for the 1252 IMGs with scores above 230 who interviewed, only 63.3% matched to internal medicine.  For the 650 IMGs with scores above 240 who interviewed, only 71.5% matched to internal medicine.

So, even if your scores are great there is only a moderately strong chance that you will get a residency in internal medicine.  Even with terrific scores you need to have the other parts of the application including clinical experience in the US with US letters.  The most important issue is not expecting to land a residency in a university program.  If you lower your expectations (even if you have terrific scores) and alter your application strategy accordingly, you may be happier in the end.

The views expressed in this post are those of Kenneth Christopher, MD and do not necessarily reflect the views of Dr. Christopher’s employer Partners HealthCare.

Low USMLE scores

A question that I was sent via email. What are my chances in Internal Medicine with scores below 85?  What can I do to improve my chances?

Not great and not much I’m afraid.  Lets look at the numbers.  From a prior post you know that 26.8% of all non-US IMG applicants will match.  Also remember that 34% of non-US IMG applicants do not interview in any field.

In internal medicine, the NRMP published data from 2009 on USMLE scores and matching for IMGs (starting on page 96).  This is a very helpful resource in determining where you fit into the matching game.

For starters, lets use the three digit score as the two digit scores are being phased out.  Now, the mean step 1 score of all IMGs (non-US and US IMGs) who match in internal medicine is 222 and the mean step 2 is in the 226 range.  (For conversion from 2 digit to 3 digit see here)

For those with a step 1 score of 200 and below, only 13.0% of IMG applicants who interview match in internal medicine.  Remember there are a lot of people who apply with step 1 scores under 200 and don’t interview so if we take those into account, the match numbers in IM for step 1 scores under 200 are terrible.  Looking at those with a step 1 score of 210 or less, only 25.6% of those who interview match in IM.  Still not great.

Lets look at step 2 scores. For those with a step 2 score of 200 and below, only 19.9% of IMG applicants who interview match in internal medicine.  Remember there are a lot of people who apply with step 2 scores under 200 and don’t interview so if we take those into account, the match numbers in IM for step 2 scores under 200 are not good.  Looking at those with a step 1 score of 210 or less, only 23.2% of those who interview match in IM.  Again not great.

I don’t interview for Family Medicine but if you look at the mean step 1 score of IMGs who match is 201 and the mean step 2 is 205.  For those of you with lower scores, Family Medicine may be “easier” to match in but has fewer residency spots.  Similar to Internal Medicine, Family Medicine is saturated with applicants.

No amount of research, publications, MPHs, or observer experiences will rescue poor step 1 and 2 scores.  Good friends of mine had great research, student clerkships, observer experiences, perfect English and connections but did not match due to low scores.

The message is that poor scores are very difficult to overcome.

The views expressed in this post are those of Kenneth Christopher, MD and do not necessarily reflect the views of Dr. Christopher’s employer Partners HealthCare.

Taking Step-3

A question from an IMG who is a US citizen: Taking Step-3 is not mandatory for me before residency. I understand that Step-1 scores are most important and Step-2 scores are a little less important. If Step-3 is not required, does taking it have a benefit on my application? I suppose if I fail the test it would hurt, but if I score a 99 vs an 85 does it make any difference?

When I look at applications I don’t look at Step 3. The AMG or US citizen candidates I see don’t have step 3 as they usually take the exam during residency. I don’t know how step 3 scores correlate to residency and ABIM board success. In a 2005 study (Academic Medicine 2005;80(10 Suppl):S21-4), investigators found associations between Step 3 scores and Step 1 / Step 2 scores (good test takers are good test takers) and PGY-l performance evaluation score (whatever that is). In my experience, it is not helpful for me to know what your Step 3 score is. On the other hand, if you fail Step 3 before you apply then something is wrong. My advice in your case is to take step 3 before residency but after you have matched. It is hard to study for step 3 during the intern year, you just don’t have the time.

Facts about Step 3 that most of you know:
Step 3 is not required for ECFMG Certification.
ECFMG Certification is required to take Step 3.
Passing Step 3 is required for an H1B visa.
The majority of states require some residency training before taking Step 3. From what I can find, Arkansas, California, Connecticut, Florida, Louisiana, Maryland, Nebraska, New York, South Dakota, Texas, Utah, Washington and West Virginia all allow IMGs to take Step 3 before residency.

I took my Step 3 in Rhode Island during residency but before fellowship. I was just happy to have a day off and not be post call. I got a nice big headache after the exam.

Step 3 scores do not matter for fellowships, as long as you have passed. If you are going to take it, make sure you study hard. The 2009–2010 Step 3 data shows that for IMGs the pass rate is only 79-83% whereas the pass rate for AMG MDs is 94-96%. Everyone tends to think that Step 3 doesn’t matter, is easy or just doesn’t have time to study for it during internship. Yes all of these thoughts are true but if you use them as excuses not to study you may not like your score…