Program Review and Contract Monitoring Protocol

Effective January 1, 1997

Overview:

The Office of Adult Career and Continuing Education Services-Vocational Rehabilitation (ACCES-VR) Centers Administration Unit provides ongoing review and monitoring of Centers for Independent Living through the review and analysis of mid year and end year narrative and statistical reports, review and analysis of annual certified financial statements and the periodic conduct of onsite program reviews.

Purpose of the Onsite Review:

The purpose of the onsite program review and contract monitoring review is to verify data used in mid year and end of year reports and to assure that Centers for Independent Living are delivering services in a manner consistent with and in accordance to:

contractual obligations;

principles and philosophy of Independent Living as defined in New York State standards;

maintenance of appropriate fiscal internal controls and practices;

performance requirements established in contract agreements.

Statutory Basis for the Onsite Review: The statutory basis for the onsite review derives from sections 1121 - 1124 of New York State Education Law.

Selection of Programs for Onsite Review:

The selection of programs for onsite review is based on a combination of criteria as follows:

indicators of contractual performance;

elapsed time since last onsite review;

geography;

size and composition of program consumer population;

consistency of program delivery with program standards and contract goals.

Procedures for Program Review and Contract Monitoring:

A. Frequency

  • Review and analysis of semi annual narrative and statistical reports by Centers Administration Unit staff.
  • Onsite review a maximum of once every five years unless otherwise indicated.

B. Notification

  • ACCES-VR compiles and publicizes an annual list of anticipated onsite reviews to be scheduled.
  • ACCES-VR provides written notification of onsite review dates 90 days prior to review for all regularly scheduled program review and contract monitoring onsite reviews.
  • Centers for Independent Living submit written documentation for preview by ACCES-VR review team liaison45 days prior to onsite review, (see Attachment A - List of Preliminary Documentation for Onsite Review).
  • ACCES-VR provides written notification of onsite review team members and onsite review agenda 30 days prior to onsite review.
  • Centers for Independent Living submit list of consumers for random selection of CSR review and consumer interviews 20 days prior to scheduled onsite review.

C. Participants on the Onsite Review Team

  • The Center for Independent Living onsite review liaison may serve as contact person and coordinates CIL staff participation during the review. Individual may be the noted signatory of the contract document or their designee. ACCES-VR Centers Administration Unit staff.
  • Other ACCES-VR staff that supervise Centers Administration or serve to provide special expertise in conduct of the onsite review.
  • Any other impartial members agreed to in advance by both ACCES-VR and the CIL.

D. Time line for Report Preparation and CIL Response

  • General findings are verbally presented by the onsite review team during an exit meeting with CIL representatives.
  • A draft written report is developed for signature of the ACCES-VR Manager of Program Development and Support Services 30 days after completion of the onsite review.
  • A signed draft report is forwarded to the CIL for review and comment 45 days after completion of the onsite review.
  • The CIL provides written comment regarding the draft report 45 days after the draft report is submitted to the CIL.
  • ACCES-VR Centers Administration Unit releases final report findings and recommendations for any relevant follow up action by the CIL 75 days after issuance of the original draft report. The written comments of the CIL are appended to the final report issued by ACCES-VR.

E. Recommendations and Corrective Action

  • ACCES-VR onsite review team members provide recommendations based upon knowledge of statewide or national best practices in the field of independent living service delivery. Recommendations/best practices are provided verbally during exit meetings and in written form designated by an asterisk. Items that fall into the category of recommendations/best practices in the protocol that follows are also designated by an asterisk.
  • Corrective action plans must be proposed jointly between a board of directors and organizational leadership. Corrective action plans require the identification of specific action to remedy deficits in performance, compliance with standards or in fiscal management and time lines for each action to be taken.  Corrective action plans are reviewed by ACCES-VR and either approved or rejected within 30 days of submission. Good faith efforts made by the CIL to submit a written corrective action plan that is deemed unacceptable by ACCES-VR will be responded to by ACCES-VR in writing within 15 days with specific technical assistance and requirements. If a subsequent revised response by the CIL within 15 days remains unacceptable, ACCES-VR may take steps toward cessation of contractual obligations. In such an instance, the CIL may appeal to the ACCES-VR Deputy Commissioner level.

Scope of the Review:

A. Review of preliminary written documentation based upon Attachment A and prior year mid and end of the year report information.

B. Verification of compliance with New York State Standards for Independent Living, (see Attachment B Key Indicators of Compliance with Standards).

C. Review of 10 percent or no fewer than 15 Consumer Service Records (see Attachment C - CSR Review Format) to determine the following:

  • Documentation reflecting requirements for maintaining a CSR is revealed in the consumer file;
  • Services are delivered in a manner consistent with Independent Living principles and philosophy;
  • A range of services is being delivered to consumers.

D. Conduct of no fewer than five interviews with consumers served by the CIL using a random selection from the set of consumers served over the past 12 months, (see Attachment D for Consumer Interview format). Interviews may occur face-to-face between consumers and onsite review team members or via telephone depending upon consumer preference. All interviews will be carried out in a manner which is non-threatening to consumers and which maximizes consumer input.

E. Board of Directors Interviews, (see Attachment E for Board Interview format). Board interviews will be conducted with a majority of board members or with up to eight individuals, faceto-face, or via telephone, individually or in a group, independent of CIL staff. Interviews will aim to:

  • Understand and assess general practices of organizational governance;
  • Commitment of the board to cross-disability representation, geographic representation and ethnic representation.

F. Measurement of Performance Indicators:

The following measures of contract performance will be discussed and assessed during the onsite review:

  • Total numbers of Persons Served During the Year as a goal in comparison to actual numbers served.
  • Achievement of Community and Systems Change Goals during the preceding contract year.
  • Results of the ACCES-VR coordinated Quality and Customer Satisfaction survey.

G. Fiscal Operations: Review of CIL policies and practices of internal fiscal management (see Attachment F Review of Fiscal Operations), including but not limited to:

  • Authorization of expenditures and/or purchasing;
  • Review of internal fiscal controls;
  • Equipment inventory;
  • Budget development.

Attachment A - Preliminary Written Documentation for Onsite Review 

Time line:

All materials noted below must be made available to the Office of Adult Career and Continuing Education Services-Vocational Rehabilitation (ACCES-VR), Centers Administration Unit, no later than 45 days prior to the scheduled onsite review.

  1. Articles of incorporation.
  2. 501 (c)3 documentation.
  3. Organizational chart.
  4. Job descriptions for each position funded under the ACCES-VR contract.
  5. Program descriptions for all programs run by the CIL.
  6. Organizational vision and/or mission statements, optional.
  7. Strategic Plan, optional.
  8. Personnel policy handbook/manual.
  9. Written grievance policy for consumers of the CIL.
  10. Written policy regarding confidentiality of customer information.
  11. Internal CIL forms used for customer tracking and data collection.
  12. Written fiscal management policies and procedures.
  13. Organizational bylaws.
  14. Written policy regarding equal access and/or reasonable accommodation for consumers and employees of the CIL.

Please note: All documentation noted above must be made available in alternate formats upon request.

Attachment B - Key Indicators of Compliance with New York State Centers for Independent Living Standards

Name of Reviewer: _____________________________

Date of Review: ________________________________

Name of CIL: ____________________________________________

Standard 1 - Consumer Control:

Key Performance Indicators:

A. Governing Board

1. Verification of consumer control:

(i) No. of persons on governing board. __________

(ii) No. of governing board members with disabilities.

*(iii) No. of different disabilities represented based upon five broad categories including: cognitive, physical, mental sensory and multiple. ___________

*(iv) No. of members from minority groups on the governing board. __________

(v) The CIL has a process for nominating and selecting board members.
____ Yes _____ No

*(vi) The nomination and election processes used allows for participation by consumers of the CIL's services.
_____ Yes _____ No

VERIFICATION SOURCE:

 

 

REVIEWER COMMENTS:

 

 

2. The board is the principal policy making body:

(i) Bylaws or other documents or practices of the CIL ensure that policy decisions are vested with the governing board?
_____ Yes _____ No
If decision or veto authority rests in a body other than the governing board, please describe.

(ii) Does the CIL bylaws specify board member roles and responsibilities?
_____ Yes _____ No

(iii) The governing board is the principal governing body of the CIL.
_____ Yes _____ No

VERIFICATION SOURCE:

 

 

REVIEWER COMMENTS:

 

 

Standard 2 - Equal Access:

A. CIL advocates for and conducts activities that promote equal access to all services, programs, activities, resources, and facilities in society whether public or private, and regardless of funding source, for individuals with disabilities. Equal access, for purposes of this paragraph means that the same access provided to individuals without disabilities is provided in the CIL's service area to individuals with disabilities.
_____ Yes _____ No

B. CIL makes available in alternative format, including braille, large print, cassette tape, electronic disk, etc. upon request, all of its written policies and materials and IL services.
_____ Yes _____ No

C. The CIL is totally physically accessible for persons with mobility disabilities.
_____ Yes _____ No

D. There are TDD/TTY's and/or other available means, either by the CIL itself or through contract, of communication to ensure access at the CIL for persons with hearing disabilities.
_____ Yes _____ No

E. The CIL is accessible to persons with disabilities requiring alternative means of ensuring ways of access to CIL services.
_____ Yes _____ No

F. State type of reasonable accommodation for individuals represented above.

G. Interpreters are available at the CIL upon request.
_____ Yes _____ No

H. Reader assistance is made available at the CIL upon request.
_____ Yes _____ No

I. A sampling of public relations materials reviewed emphasizes attention to equal access to society for all individuals with disabilities.
_____ Yes _____ No

VERIFICATION SOURCE:

 

 

REVIEWERS COMMENTS:

 

 

Standard 3 - Consumer Service Record:

The CIL maintains the following information in each Consumer Service Record:

A. Annually updated educational and employment status data from intake/consumer profile forms.
_____ Yes _____ No

B. Consumer signed and dated notification of rights and responsibilities as a consumer of the CIL.
____ Yes ____ No

C. Includes acknowledgement of consumer right to confidentiality. Rights and responsibilities document.
_____ Yes _____ No

D. Rights and responsibility document acknowledges and describes an internal CIL grievance process.
_____ Yes _____ No

E. Rights and responsibility document acknowledges the availability of ACCES-VR with a contact address and phone number for addressing consumer complaints about CIL services.
_____ Yes _____ No

F. There is a record of specific services provided including:

1. Specific services received _____ Yes _____ No

2. Dates of service contact _____ Yes _____ No

Referrals made on behalf of consumers (either written or other formal referrals).

VERIFICATION SOURCE:

 

 

REVIEWERS COMMENTS:

 

 

Standard 4 - Community Options and Community Capacity:

In the reporting year, the CIL promoted the increased availability and improved quality of community based programs that serve individuals with disabilities and promoted the removal of any existing architectural, attitudinal, communication, environmental or other type of barrier that prevents the full integration of people with disabilities into society. During the reporting year, the CIL performed at least one activity in each of the following categories: (please specify verification source document or activity).

(a) Community Advocacy. ACCES-VR approved year end report.

Document/Activity:

 

(b) Technical Assistance to the Community on making services, programs, activities, resources, and facilities in society accessible to individuals with disabilities.

Document/Activity:

 

(c) Public Information and Education. ACCES-VR approved year end report.

Document/Activity:

 

(d) Outreach to populations of individuals with disabilities that are unserved or underserved in the CIL's outreach plan.

Document/Activity:

 

VERIFICATION SOURCE:

 

 

REVIEWERS COMMENTS:

 

 

Standard 5 - Resource Development:

The CIL during the reporting year conducted resource development activities to obtain funding from sources other than New York State Independent Living Aid to Localities funding.

VERIFICATION SOURCE INCLUDES:

A. End of Year CIL Annual Performance Report.
_____ Yes _____ No

B. Annual Certified Financial Statement.
_____ Yes _____ No

REVIEWER COMMENTS:

 

 

Attachment C - Consumer Service Record Review Format

Name of Reviewer: _____________________________

Date of Review: ________________________________

Reviewers complete this form for each CSR reviewed to determine compliance with Standard 3.

Name of Consumer: _________________________________________

The CIL maintains the following information for each Consumer Service Record:

Check Yes or No

1. Educational status and employment status data on consumer intakes/profiles are updated annually.
_____ Yes _____ No

2. There is a document describing rights and responsibilities as a consumer of the CIL.
_____ Yes _____ No

A. Document is signed by the consumer.
_____ Yes _____ No

B. Document is dated.
_____ Yes _____ No

3. The document describing rights and responsibilities as a consumer of the CIL describes the right to confidentiality.
_____ Yes _____ No

4. The document describing rights and responsibilities as a consumer of the CIL outlines the internal appeals process to resolve complaints with the CIL.
_____ Yes _____ No

5. The document describing rights and responsibilities as a consumer of the CIL provides notification to the consumer of the availability of ACCES-VR to consider consumer complaints about services.
_____ Yes _____ No

A. ACCES-VR Contact office/person is identified.
_____ Yes _____ No

B. ACCES-VR consumer complaint phone number is provided.
_____ Yes _____ No

C. ACCES-VR address is provided.
_____ Yes _____ No

6. The CSR identifies specific services received.
_____ Yes _____ No

7. The CSR identifies dates of service contact.
_____ Yes _____ No

8. The CSR includes relevant referral information when appropriate, provided by the CIL on behalf of consumers.
_____ Yes _____ No

9. If a release of information is present in the CSR, then it contains appropriate and legal information.
____ Yes ____ No

General Comments:

 

 

Attachment D - Consumer Interview Format

Reviewers develop a profile of responses based on each consumer interview. The feedback remains anonymous.

Reviewer's Name: _____________________________________

Date of Interview:____________________

Consumer's Name: ____________________________________

1. How long have you been a consumer at the Center?

 

 

2. How did you originally hear about the Center?

 

 

3. Can you tell me about the services you have received at the Center?

 

 

4. Please describe ways you may have benefited from the services you received?

 

 

5. How would you describe the level of skills and knowledge of staff you have worked with at the Center?

 

 

6. Did you always feel in charge of your services at the Center?

 

 

7. Are you aware of services that the Center provides in addition to the services you have received?

 

 

8. Would you or have you recommended the Center to others?

 

 

REVIEWERS COMMENTS:

 

 

Attachment E - Board of Directors Interview Format

Reviewer(s) Present at the interview: ___________________________________________________________________________

Date of Interview:__________________

Board members present at the interview: _________________________________________________________________________

Board interviews are conducted either individually or as a group. Information provided is summarized in a composite overview with appropriate recommendations or compliance requirements indicated.

1. How long have you been on the board?

 

 

2. How were you selected to be on the board?

 

 

3. Please describe the methods used at your Center to nominate and appoint board members.

 

 

4. Is the board's membership representative of the community served by the Center in relation to:

A. Geography _____ Yes _____ No

B. Ethnicity _____ Yes _____ No

5. If any aspect of the previous question is negative, please describe some steps under consideration to enhance representation.

 

 

6. How many board members have disabilities and is cross-disability representation achieved?

 

 

7. Please describe the board committee structure and explain how it works.

 

 

8. What is the board's relationship to Center staff, programs and initiatives?

 

 

9. How do you know what is happening to and with consumers and community and systems change advocacy efforts?

 

 

10. Are you now, or were you a consumer of CIL services in the past?
_____ Yes _____ No

11. How does the Board evaluate the Executive Directors performance?

 

 

12. What is the board's vision for the CIL over the next three years?

 

 

13. Do you have any questions for ACCES-VR?

 

 

Attachment F - Fiscal Management, Budgeting and Equipment Inventory

Name of Reviewer: _____________________________

Date of Review: ________________________________

Name of CIL: ____________________________________________

1. An individual has been designated to have clear responsibility to assure that contract funds are properly used.
_____ Yes _____ No
Title(s) of designee: _____________________________

2. Accounting records including cost accounting records are supported by source documentation.
_____ Yes _____ No

3. Required reports accurately account for the CIL's funds.
_____ Yes _____ No

4. Procedures are followed to assure expenditures are consistent with the approved budget of the CIL.
_____ Yes _____ No

5. ACCES-VR has approved any budget revisions prior to expenditure of funds.
_____ Yes _____ No

6. The CIL has written procedures that minimize the time elapsing between the transfer of funds and dispersement of funds.
_____ Yes _____ No

7. The CIL has a method for recording time in attendance of staff.
_____ Yes _____ No

8. All financial status reports both of an internal nature and those required by the grant agency are current and accurate.
_____ Yes _____ No

9. VERIFICATION SOURCE:

 

10. The financial status report reflects program income, cost sharing, matching indirect cost rates.
_____ Yes _____ No

11. The CIL distributes administrative costs among its various funding sources in an equitable manner as might be found in a cost allocation plan.
_____ Yes _____ No

12. If yes, and if a cost allocation plan has been developed, provide a copy of the plan (this is useful for future budget adjustments and is of value when providing technical assistance to other CIL's).
_____ Yes _____ No

13. The accounting system is computerized.
_____ Yes _____ No
If so, what software is used. ____________________

14. The elements of the formal accounting system are:

A. General ledger ________

B. Grant ledger ________

C. General journal ________

D. Cash receipts ________

Other (please specify)_____________________________

15. The formal accounting system is maintained by

A. Cash basis ________

B. Accrual basis ________

Other (please describe)____________________________

16. Purchasing and payment functions are separated.
_____ Yes _____ No

17. Payments are made only if dispersements or checks are countersigned.
_____ Yes _____ No

18. All funding sources are accounted for separately.
_____ Yes _____ No

19. Expenditures are recorded by budget cost category and funding sources.
_____ Yes _____ No

20. All accounting records are supported by accurate documentation and entries are made on a regular basis.
_____ Yes _____ No

21. Payment vouchers are identified as to grant number, expense classification and transaction date, and funding sources.
_____ Yes _____ No

22. The CIL maintains current and complete records of all property purchased with contract funds.
_____ Yes _____ No

23. The CIL has a system for controlling inventory (general ledger control account card reports, and property labels).
_____ Yes _____ No  

24. The CIL keeps duplicate reference.
_____ Yes _____ No

25. Maintenance of property and equipment records are kept.
_____ Yes _____ No

26. Records contain: descriptions of each piece of equipment; purchase date/cost; manufacturers serial number, model number, federal stock number, national stock number, or other identification number; location and condition of the equipment, date information was recorded; and ultimate disposition.
_____ Yes_____ No

27. The CIL inventories property at least annually.
_____ Yes _____ No

28. Date of last inventory: ____________________________

29. Adjustments to inventory accounts are made only on written authority of a designated official.
_____ Yes _____ No

30. Name/Title of designee: ___________________________________________________________________________

31. The CIL guards against loss, damage or theft of property.
_____ Yes _____ No

32. The CIL has policies, procedures and controls for purchasing and disposing of property.
_____ Yes _____ No