Wednesday, October 31, 2012

More on proprioception


 A couple posts back I blogged about a new tool that has been in development called the Comprehensive Observations of Proprioception.  I was a little surprised about the editorial decision to publish an article about performance on the tool without publishing about the tool itself.  Now we have a paper on the tool itself - so the ordering of publication is a question for AJOT editors - not the authors of the paper.

The authors describe the tool as an observational measure that is criterion referenced.  The test includes 18 items that purportedly represent some aspects of proprioceptive function and they use literature review as one tool to substantiate the content validity of the items.  As I mentioned in the original post on this matter I am concerned that some of these items might represent some aspect or measure of proprioception but then again they also might not.  Fully 25%+ of the items are behavioral measures like 'overactive' and 'enjoyment when being pulled' that are very unclear to me if these are functional and discriminatory measures of a proprioception construct.

Use of content experts to establish validity is a well-established mechanism in item development.  However, the authors report disregarding the content experts on some items because even though the ratings of experts did not achieve a degree of criticality the authors believed that the items should be included because they are represented in the literature.

Some items that the experts excluded were curious.  They excluded 'Muscle tone is hypotonic' but they included 'Decreased muscle tone.'  This is very difficult to understand without having the benefit of operational definitions of all these items.  A Likert scale was reportedly used to rate item performance from (1) typical performance to (5) most severe form of proprioceptive processing difficulties.  If this is a criterion referenced scale, I was left wondering what the cutoffs are on these measures and how they operationally defined mild or moderate forms of proprioceptive processing difficulties, particularly on the 18 items listed.  The authors report that the test can be taught to a therapist with a minimum of two years of experience after receiving a brief training.  I am struggling to understand if there are operational definitions for this Likert scale establishing discrimination between ratings for these very vague behavioral observations - and if there are - how would it only take a brief training to reliably rate a child on this tool?  It seems that if there were true criterion points that this could be very complicated.

This leads into a significant limitation in this tool that has to do with relying on a constricted set of therapists who are working in a common clinic or set of clinics.  This is nothing new - I talked about this in 2009:  " It is a pertinent issue - you can't talk about broad validity of constructs when you only include people who are all drinking the same Kool Aid, so to speak."  This is the criticism that you will receive when you use convenience sampling, which the paper rightly identifies, but it still underscores a really big problem.

Additionally, the tool would be strengthened considerably if an evaluator was blinded to diagnostic condition before running comparison analysis between children with known problems and typically developing children. 

It would also be strengthened if it didn't use parent report measures as a measure of criterion validity.  For example, is there any surprise when this test correlates with the Body Awareness section of the Sensory Processing Measure?  If one test asks a parent to comment on "Bumps or pushes other children" and the other test includes observations of "Crashing/Falling/Running" then what else would we expect? 

Finally, the test also relies on the Kinesthesia subtest of the SIPT - and this is a major flaw because the value of any factor analysis is only as good as the data going into the analysis.  Everyone knows that the KIN subtest is weak and unreliable - so why even use it as a point of criterion validity?  There are items on the SIPT which would seem to be arguably stronger measure points of proprioceptive processing, such as the eyes closed measures on the SWB subtest, or maybe even including Oral Praxis.  Why include the weakest proprioceptive measure on the SIPT?  I am generically aware of some work done by graduate students at Brenau University last year about pilot revision of the Kinesthesia test but I don't think they published yet (??).  What I am saying here is that I know I am not the only person on the planet to think that the KIN test is pretty flawed.  Garbage in, garbage out - so they say.  Makes it hard to know what to do with this factor analysis that relies on the weakest of all the SIPT subtests.

So there is my blunt analysis - and I am attempting to be as constructive as possible because when researchers go and do things they probably would benefit from end-users who read their work and are trying to see how to use it.  Is proprioceptive processing an important construct?  It sure is.  Do kids who have difficulties sometimes have difficulties with proprioceptive processing?  I believe they do. 

But to me I think this tool is too weak.  Maybe I would feel differently if I saw operational definitions for these items... and operational definitions for the Likert Scale and some understanding how how cutoffs were determined.  I would also like to see evaluators blinded when they are looking at discriminatory item analysis - and I would like to know that content experts were broadly represented so there was no danger of everyone thinking with a regional mindset. 

I look forward to more, so we can continue to move these kinds of ideas forward.

References:

Blanche, E.I., Bodison, S., Chang, M. & Reinoso, G., (2012).  Development of the Comprehensive Observations of Proprioception (COP): Validity, reliability, and factor analysis, American Journal of Occupational Therapy, 66, 691-698.

Thursday, October 18, 2012

Support regulations to remove potential conflicts of interest in the New York State Early Intervention Program

More regulatory mumbo-jumbo, while I am on a roll:

The New York State Department of Health has proposed an amendment to Subpart 69-4 of Title 10 of the New York Codes, Rules and Regulations, the Early Intervention Program. The public comment period ends October 22, 2012.  The new regulations create a requirement for arms-length relationships to tamp down conflict of interest that may be contributing to over-utilization and cost over runs.

An arms-length requirement between evaluators and providers is reasonable given the evidence of inappropriate utilization and significant cost over runs in these programs.  Although not all inappropriate utilization can be attributed to this factor, it is ethically correct for providers to remove any possibilities of conflicts.  Existing rules and regulations, whether in professional practice acts or the EI regs themselves, have not controlled this problem

Here is a pertinent part of the regulation: 

 (ii)(a) For children referred to the early intervention program on or after December 1, 2012, or for children referred to the early intervention program prior to December 1, 2012 for whom an additional evaluation or partial evaluation is requested on or after December 1, 2012 for the purpose of adding a new service, neither the evaluator which conducts an evaluation of a child, an approved agency which employs or contracts with the evaluator, nor a relative or business associate of the evaluator, shall provide early intervention services to such child unless authorized by the commissioner, after consultation with the early intervention official, due to special circumstances related to the evaluator’s qualifications or availability or other extraordinary circumstances in which there is a clear showing that the child will not be able to access needed services absent such authorization.

(c) If the commissioner finds there is a shortage of evaluators or approved providers in certain disciplines in a particular region of the state, the commissioner may issue a standing authorization, on such terms or conditions as he or she deems appropriate, which shall remain in effect in such region until such time as the commissioner determines that such shortage no longer exists. 

Studies completed by VESID clearly demonstrate a pattern of self-referral conflict of interest when the same therapist does evaluations and intervention (not just for  OT, but all professions). This is most notable in NYC in transition from EI to CPSE. In a 2007 study of the statewide preschool cohort, the numbers of children identified as needing services during transition from EI to CPSE were 73% in NYC; 41% in Large Four; and 34% in other districts. NYC has the largest percentage of conflicting incidences, which is why it was named specifically in the regulations as having the greatest impact in that area.

Certainly there are other 'big city' factors that might lead to greater 'identification' when transitioning to CPSE but you still can't escape the face validity to the concern that a potential conflict of interest may exist. Ethical practitioners should not be concerned about this requirement. Exceptions are allowed in the regulations when staffing or other barriers make it difficult to avoid the conflict - but the value itself to respect the potential of a conflict of interest is valid.

Many therapists are upset about this proposed regulation; the New York State Occupational Therapy Association and other groups have written letters opposing the regulation.  However, there is a long term game to consider in all this and in NY specifically our tax-supported care systems are strained to an absolute maximum. Our ability to engage people who need our services through municipal systems is already threatened by ever-changing Medicaid rule complexities, constricted staffing requirements, bureaucratic inefficiencies that fail to capture reimbursements, and many other factors. If the programs are going to survive then we should be at the table and we should be making honest attempts to promote quality and to control costs in those areas where legitimate concerns are presented. That doesn't mean we are promoting governmental interference; rather, we should be at that table and helping to shape that policy so that we can actually fix these broken systems.
 

 The need to reform our systems is obvious.  Therapists are losing jobs, reimbursements are decreasing, caseloads are increasing, and requirements for documentation become more burdensome each day.  We will not positively contribute to reform if we refuse to accept reasonable regulations like this one.

Or we can just go kicking and screaming into the night, refusing to budge and refusing to compromise. This system has to change - and it will change whether people participate productively or if they simply attempt to obstruct. 
There is no harm done by a regulation to promote an ideal of removing all potential conflicts of interest when we are working within municipal/tax-funded systems of care.  I see no problem whatsoever with this new requirement.  Quite frankly, I find it embarrassing to see the state associations and other professional groups opposing this.

You have until 10/22/12 to register your comments to the State: 

Katherine Ceroalo
New York State Department of Health
Bureau of House Counsel, Regulatory Affairs Unit
Corning Tower Building, Rm 2438
Empire State Plaza
Albany, NY 12237
(518) 473-7488
(518) 473-2019 (FAX)
REGSQNA@health.state.ny.us



References:

A longitudinal study of preschool special education: Final report. (September 6, 2007).  Retrieved from http://www.p12.nysed.gov/specialed/preschool/study/

Proposed regulations: NYS Department of Health.  Retrieved from http://w3.health.state.ny.us/dbspace/propregs.nsf/4ac9558781006774852569bd00512fda/9cbdd5b7692c1c9c85257a6a0054f4fd?OpenDocument 

Wednesday, October 17, 2012

Different perspectives on concerns with CPSE services in NYC


The following material was copied from the NYSOTA Facebook page.  A fan of the page (George Nickel) posted a general "calling out" to NYSOTA which prompted my response.  This might be lengthy, but I believe that it is instructive.


Post from George Nickel on Facebook:


Okay, I am calling you out. We have received written support from The New York State Speech Hearing and Language Association and the Regional Physical Therapy Association but none from the OT regarding the issue of the Related Services Tier System of The NYC Department of Education. Is it not an important issue that children with special needs and their families need you to join with the other organizations to advocate for not just them but for your profession?

Christopher Alterio responds:

George, this sounds like a local RFP that you lost and not a professional problem. if you have some more detailed information to share I would be interested in seeing it. I looked at various Facebook pages that are linked off of your page and I also reviewed the information on the United EI Providers site about CPSE services. From what I can tell there was an RFP process. Some people won and some people lost. It seems that the transition is bumpy and some kids are allegedly not getting services. There is no way to confirm these allegations and the DOE has not issued a response that I can find.

Why should a state association meddle in a local RFP process? When I bid on contracts sometimes I get them and sometimes I don't. Transitions between agency providers is sometimes rocky, both when starting a contract and sometimes when ending the contract. I never considered calling on the state association to intervene on behalf of a local RFP process just because I lost. That is just business - and it is up to people to be competitive or to re-tool themselves if they want to continue.

There is no doubt that the system is over-stressed and that is precisely why you are seeing competitive bidding for service provision. That is also why you see parents just bailing on the system and getting services privately.

So I am just your counterpart on the other side of the state, and I don't ever hear providers around here ask NYSOTA to save them when contracts are competitively awarded and service providers are changed. Help me understand more if there is some larger issue at play. Why should a state association be involved in this if it is a local RFP and some people lost??

George Nickel responds:

Chris you have assumed a great deal but missed the important facts.First, there are children not receiving services and the primary agencies are not sending children through the tier system in a timely manner. Secondly, many of your organizations members fees are being brought down by agencies bidding for lower fees.Further, most agencies are not taking the same percent it cuts but passing it directly to therapists.Thirdly,when a system promotes "too big too fail"/monopolies and small private practices to close that is not business as usual. Lastly, I asked you to join in advocacy for the children and their families, however by your response you seemed to have targeted the wrong issue. What is that about? And you checked my links,what is that about? Why didn't you call some of your members and find out from them directly what is going on?


Christopher Alterio responds:

 George, just to be clear (and to also ease NYSOTA's liability concerns!) I am not a NYSOTA member and I don't represent NYSOTA's position. I am just an OT who will probably become a member in the near future based on some good progress that I understand NYSOTA is making on some previously identified concerns - all beside the point.

Anyway, you are asking for support and I am always interested in professional issues so I am asking for details. I looked at your links, on some blogs, and at the United EI Provider site because these are just about the only place I could find any information on this topic. Not surprisingly, there is no information to be found on the NYC DOE website or on some of the websites of large agencies that were listed in some of the blog posts I found. I actually was going to call the NYC DOE for information but their website is a maze and there is no place to call!

If children are not receiving services then of course that is a problem but like I said previously - transitioning and service coordination when contracts change can be a nightmare, and I can only imagine how complex the problem is in a system as large as NYC. So is this a structural problem where NYC DOE is purposely denying services? There are remedies for that, obviously via due process proceedings. If there is evidence that there is purposeful structural manipulation to deny care then of course that would seem to be a large issue for a professional member association to weigh in on. I am just asking for the evidence.

Instead of evidence that this is a purposeful structural manipulation most of what I can find online are concerns like yours - that small providers have lost contracts and are out of work because of a competitive RFP process. If rates are cut because someone in the marketplace is able to provide the care at that rate then I suggest that is just the free market at work. Now if over time that care provided at a lower rate does not meet standards or needs then the free market will correct itself.

I just don't understand the outrage. These are municipal funds being used to pay for these services and the municipality has a fiscal responsibility to be judicious caretakers of funds. As was noted by a blog commenter on this topic - the municipality is responsible for providing FAPE and appropriate care is not the same as 'Cadillac' care. The legal system has been trying to find a line on what is 'appropriate' for a long time and the push and pull between cost and value is an old old debate. This is nothing new.

So this does not get a knee jerk response of support from me. As many people in our community know I am generally the last person to stand up and believe that the municipalities are blameless. However, if you are looking for support, how about some more information and evidence??

Just to reiterate, I don't represent NYSOTA and this is my own opinion. It sounds like people are asking for a membership organization to solve concerns that some people have about a local NYC RFP where there were winners and losers. That almost sounds like a call for a quasi-Union - and I will be following this rather closely because I am not of the opinion that a state association should be trying to function as a Union and to restrict competition in a free market where many therapists (self employed OR in large agencies) are all supposed to be on a level playing field. If the NYC DOE is doing something structurally incorrect then that is fine. But if NYSOTA is supposed to support one group of therapists over another just because someone lost in a fair bidding process then I don't agree at all.

George Nickel responds:

Sorry, but I thought you were a NYSOTA Board Member weighing in, thanks for your input. Perhaps you need to do more research on the issue. You are welcome to join conversations at SI Parents Therapists Unite on FaceBook and perhaps one of your colleagues would respond to your questions. As far as free market, again there are dangers when individual private practices are threatened to join large agencies or face losing work or their practices. This is even more important in today's economy and our country relies on small businesses. I am in favor of those agencies having their roots in our community and not being from out of state or country. I am against greed and as one of my mentors has thought me, "take enough for yourself and leave the rest for others". Not such a bad practice, now is it? In a health related field we should be networking more and competing less that is what would work best for the individuals we provide our services to in our community. Further, We educate and advocate for our professionals and those we provide our services for in our communities.

Christopher Alterio responds:

Research = reading everything to be found on the Internet including Facebook pages, websites, blog posts; scanning news services for articles on this topic, engaging in online discussion forums, and asking for those concerned to provide more details. Is there anything I am missing?

What are the 'dangers' you are talking about? Unless NYC has some different ruleset that I can't locate, municipalities the size of NYC are required to keep a list of approved service providers and the municipalities are responsible for determining the appropriate rate for those services.

Where is the 'greed' if an agency can provide a service at a lower cost or if labor supply and the economy dictates that the reimbursement point is lower than you might like? Are these large agencies greedy just because they won a competitive bidding process? I don't understand.

Lack of competition breeds inefficiency. Inefficiency contributes to waste. According to the US House of Representative Committee on Oversight and Governmental Reform in a report earlier this year, "these school based (Medicaid) services, which cost taxpayers $800 million annually and account for 44% of national Medicaid spending of this type, have created a Medicaid monster in New York schools. A number of audits in recent years indicate that 86% of Medicaid claims paid to New York City schools from 1993 to 2001 either lacked any explanation as to why the services had been ordered or violated some other regulation or requirement."

All of us as school based providers are acutely aware of the State Plan Amendment and the tremendous pressures everyone is under because of the lack of fiscal responsibility and in fact the outright fraud in these programs. I don't know about anyone else but I would like to see school based services available far into the future - but if we close our eyes and fail to acknowledge the HUGE problems in front of us we will serve our short term interests about our own salaries but fail to appreciate the long game and whether or not we are contributing to a real solution.

I am a small private practitioner. I have won and lost contracts. I also saw the Medicaid train wreck and before it hit I re-tooled my practice and oriented it toward direct access and private insurance. I still participate with municipal contracting and still recognize that we all need to be more fiscally sensitive and contribute to a long term solution. I don't see how attempts to quasi-unionize the workforce and eliminate competitive bidding brings us any closer to solving the large problems.

Then again maybe I just don't know much and need to keep researching. But I don't think so.

Follow the fun on the NYSOTA Facebook page:  http://www.facebook.com/MyNYSOTA