Thursday, June 23, 2011

The lost art of evaluation in pediatric occupational therapy

This is a topic that is probably long overdue - it is something that I have observed in my geographic area for a long period of time. Based on conversations I have with therapists around the country I know that it is not exclusive to my area.

Increasing demands on therapist time, decreasing reimbursements, and dependence on a non-centralized workforce that is not subject to an intense quality improvement process has contributed to significant changes in how occupational therapy 'evaluations' are completed.

Our agency made attempts to impact this system while contracted to complete some evaluations, but 'contractor status' did not place us into a position to make broad system changes. Now it seems that the entire community is stuck into a cycle of low expectations based on long history. Unfortunately, there is no centralized service delivery system or 'top-down' method of assuring quality control - so I think the only way this can change is for the stakeholders to start increasing awareness of the problem and having a conversation.

Both therapists and parents should be interested in this issue. Therapists should be interested because I am calling the quality of our collective 'evaluation' work into question. Parents should be interested because they should know what to expect out of an occupational therapy evaluation for their children.

The American Occupational Therapy Association publishes standards of practice that outline some basics that should be part of all evaluations - but these standards are not very specific. Many evaluations that I see completed as a part of preschool and school-based practice do not even meet these basic standards. Currently, many evaluations that I am seeing can be outlined as follows:
  1. Very general statement regarding the child, including name, classroom, and very minimal if any history
  2. Long boilerplate descriptions of standardized tests that were used, followed by a very brief report of the child's performance on those tests
  3. A 'summary' section with minimal if any analysis and deferred decision making to the CPSE or CSE regarding eligibility for services.
I understand how 'evaluations' have devolved to this state - there are significant time demands and therapists do not have the time to complete lengthy evaluations. Also, there is no demand from CPSE or CSE for higher quality because the bottom line from the committee perspective is 'are children eligible or are they not eligible.' The committees are not in business to promote best practice; they are in business to determine eligibility. If a committee is receiving an evaluation report from a therapist it is generally making a de facto assumption that the evaluation being handed to them is appropriate and complete. Committees occasionally receive feedback about quality because some parents identify that the 'evaluations' are inadequate - and that causes external data (like a private evaluation) to be brought to the table.

External data in the form of a private occupational therapy evaluation causes a lot of consternation to committees, who don't understand 'what is wrong' with the evaluations completed within the system. That causes committee chairs and their committees to be puzzled at what is documented in a private assessment - and contributes to the general sense of 'that's not the way that we do things in schools' or even more commonly 'that is a clinical evaluation and not a school-based evaluation.' The most discouraging part of this problem to me is the engendered culture of low expectation and how practitioners create these myths about what constitutes acceptable practice.

The role of related service providers and their relative power within schools also contributes to the problem; this group of professionals is generally not in a position to take on the systems where they work and they are not always likely be on the front lines of change promotion within their systems.

Anyone who has been involved in this system for any length of time can identify with this issue.

So what can we do to change this?

I don't want to talk about 'best practice' because the term is overused and I also don't think it represents an intermediate step that we can reasonably take to improve. Let's talk about 'better practice.'

'Better practice' means taking a step forward from where we are now. It might not be best, but it is moving in that direction. From our current position, I propose the following for 'better practice' occupational therapy evaluations completed for CPSE and CSE:

  1. Background information including reason for referral, identification of medical issues, and lists of allergies and medications. Birth and developmental history need to be present, including history of CPSE or CSE involvement. The inclusion of developmental and medical history does not make an evaluation 'clinical.' This is basic information that is required to form a contextual understanding of the child's performance difficulties.
  2. Description of the child's ability to participate in the assessments. This also provides important contextual understanding of the results.
  3. A LISTING of the assessments used. If there is a value to 'explain' the tests for the parent audience then provide the parents with a separate sheet of paper with that information. The evaluation should never be 90% boilerplate explanation of what tests were administered.
  4. Direct performance observations AND performance on testing. Organize observations and test data into logical performance categories such as 'Physical skills,' 'Sensory skills,' 'Cognitive skills,' 'Regulatory skills,' and 'Social/emotional skills.'
  5. Apply this to actual function in their environment, including observations of how these performance attributes impact participation in personal care, learning, and play or socialization. Here it is likely that school based therapists will limit the 'environment of concern' to the school setting, which is appropriate.
  6. Summarize the findings, identifying areas of strength and areas of need. Form a summary opinion of what is happening with this child's life and make referrals for other services as needed.
  7. MAKE AN ACTUAL RECOMMENDATION! There is nothing wrong with giving your professional opinion. It is up to the committee to accept or reject your recommendations. That DOES NOT MEAN that you defer recommendations to the committee. This is where many committees fail - because it is absolutely fine for professionals to make recommendations and then for a committee as a whole to review those recommendations and decide what is most appropriate. For example, you may identify that a child's needs are so severe that you recommend OT three times a week. However, you may get to the committee meeting and find out that colleagues in PT, speech, and education made similar recommendations for their domains and in total it would be 'too much' for the child to tolerate in their day. The committee may then consider that a different level of service or an altered service delivery method is needed in consideration of ALL the data on the table. That is fine and is actually the STRENGTH of multidisciplinary planning.
  8. Follow up with the parents and teacher and talk to them on a regular basis so that you are not providing or documenting this service in a vacuum.
These are just some preliminary ideas. I am interested in feedback - again with the intention of promoting 'better practice.'

If we take some solid first steps, wouldn't it be nice to have a conversation about 'best practice' next?

Wednesday, June 15, 2011

NYS Early Intervention Update - SEICC meeting

The State Early Intervention Coordinating Council held a meeting on June 14 (yesterday) to accept or reject the new regulations regarding issues including the 15 minute increment billing and adjusted early intervention rate calculations.

I just finished speaking with Holly Kennedy from the EIP and she confirmed that the current rulemaking was rejected and that there will be a new rulemaking (look for the new Rules proposals in the Register around the end of June).

There will be another 45 day comment period (but no public hearings). Ms. Kennedy stated that some alternate proposals may appear in the rulemaking but she was not free to be specific about those at this time. I have some initial information about those proposals but won't publish my information until I can verify and source it.

Finally, I was told that any current service plans will continue in effect past any new rules - and will be revised on an individual basis during the regular review period.

When I get details about the new rulemaking I will post them here or if they are brief enough - on our Facebook page.

My apologies that this is more fact than analysis - but at this point I am figuring that anyone interested in this particular topic understands the issues. If there are specific questions I can address about any of this please email or comment.

Stay tuned.

UPDATE: 6/21/2011

I wanted to provide a link to the full meeting. It is rather lengthy, but for those who are inspired I recommend watching the section on revised rates, the section on proposed revisions (the 15 minute increment issue), and the public comment. You can access the video at

The EIP party line on proposed rate changes is that they 'can't comment' on them because they have not yet been proposed. The good news is that they scheduled some public forums to gather feedback but the bad news is that they scheduled the release of the regulation revisions AFTER the meetings.

The EIP reports that they expect to realize 6.2 million dollars in savings in FY 2012-2013 but that these savings will not represent significant changes in reimbursement. I believe this can be said with a straight face because they are operating under a general assumption that a 45 minute session length is standard. Of course it is not - as was expressed quite vociferously in the public comment period on the video.

I believe that it is dishonest to state that this level of funding cut won't be significant. I also believe that it is fundamentally dishonest to schedule public forums before the proposed changes are announced.

The jaw-dropping moment of the video happens around 182:53 when EIP Executive Director Brad Hutton stated, "I think its fair to say that actually these proposals came out of discussions at the RAP [Reimbursement Advisory Panel]... we felt like the genesis was the general public of external stakeholders." Someone needs to tell Mr. Hutton that with all the legitimate questions about following the NYS Open Meetings Law, the significant challenge with obtaining minutes of the RAP meetings, and believing that providers helped generate these back-door reimbursement cuts qualifies this statement as the Pinocchio Story of the Year.

Watching this video will help you understand that our government programs are inefficient, poorly managed, and unable to be financially maintained with all of the perpetual bureaucratic bloat and mandate.

Redux for practitioners: Continue to prepare for sea change in how you deliver your services.

Redux for families: Begin to plan for new ways to meet your children's needs.

Thursday, June 09, 2011

The news is so sad today I don't even know what to say.

There are a couple important stories in the news today that everyone needs to watch.

In this first story we see an absolute lack of understanding in how a governmental agency responded to an adult who was trying to board a plane with his family:

Dr. David Mandy: Special Needs Son Harassed by TSA at Detroit Metropolitan Airport:

In the second story we see people completely disrespecting and interfering with a Special Olympics activity. It is certainly within their rights to express their displeasure about whatever they want, but sane people would choose a time and place that is appropriate for such a protest:

Lastly, and most tragically, is the continued failure of NYS - all these years after Willowbrook we thought it would never happen again:

It is very clear to me that as a society we still have a very long road to travel.