����������

To establish that organisations are meeting the relevant standards the International Standards Organisation (ISO) has developed standards relating to quality management systems. They define quality and quality systems as (2): 

The Requirements of the Standards for Registered Training Organisations 2015

Clause 2.2 in in the Standards for RTOs focuses specifically on RTO improvement and the quality system that drives the outcomes of its business. It indicates that the RTO:

• systematically monitors the RTO’s training and assessment strategies and practices to ensure ongoing compliance with Standard 1; and
• systematically evaluates and uses the outcomes of the evaluations to continually improve the RTO’s training and assessment strategies and practices. Evaluation information includes but is not limited to quality/performance data…validation outcomes, client, trainer and assessor feedback and complaints and appeals.

The requirement in Standard 2 goes well beyond the quality of training and assessment to include all aspects of the RTO. The Glossary in the Standards for RTOs defines ’operations’ as “including training, assessment and administration and support services relating to its registration, including those delivered across jurisdictions and offshore”.

The context statement for Standard 2 states that “The RTO is ultimately responsible for ensuring quality training and assessment within their organisation and scope of registration, regardless of any third-party arrangements where training and/or assessment is delivered on their behalf,” and concludes by observing that “evaluating information about performance and using such information to inform quality assurance of services and improve training and assessment is sound business and educational practice. The information used to evaluate RTO performance must be relevant to the operating characteristics and business objectives of the RTO and will vary from one RTO to another.”

There is no uniform strategy for quality assurance, and RTOs are expected to develop their own processes to reflect their clients, industry, operational modes, and objectives.  Internal audit is one process that an RTO can use to test the effectiveness of their systems and processes.

Audit Principles

The Standards for RTOs have at their core four main drivers which reflect modern regulatory practice:

• shifting the balance of audit and assessment from a focus on process and inputs towards the monitoring and measuring of outcomes;
• building in a stronger emphasis on continuous improvement;
• developing a risk management approach which uses a range of indicators ‑ including outcomes performance measures. This range of indicators will inform the scheduling and depth of audits. At the same time, risk management will support a more streamlined audit system for RTOs with high quality outcomes; and
• incorporating the option of assessment against higher level performance criteria for RTOs.

These drivers have determined the principles that underpin regulatory audit practice undertaken by TAC and are equally applicable in internal audits.

• Systematic
• Outcomes Focused
• Evidence Based
• Flexible
• Continuous Improvement Focused
• Fair, Open and Transparent

Systematic

Audits are conducted in a systematic manner based on an audit sampling strategy to ensure that the audit findings, conclusions, and recommendations accurately represent the organisation’s operations. Audits are planned with all staff involved in the audit and aware of their roles and responsibilities.

Outcomes Focused

To focus on outcomes, the audit process reviews evidence provided by the RTO about what has been achieved against the Standards for RTOs. There is no “one-size-fits-all” approach to compliance – the audit outcome will be based on whether the RTO’s systems are working as intended and meeting the requirements of the Standards.

The auditor will consider whether:

• the RTO’s evidence has met the requirements of each Standard or clause; and
• results were achieved through the planned and systematic deployment of specific actions taken by the RTO.

Evidence-Based

Audit findings in relation to compliance with the Standards for RTOs are based entirely on the evidence available to the auditor. Judgements about whether evidence demonstrates compliance are guided by consideration of these questions:

• Is the evidence valid - is it reflecting the requirements identified in the Standard or clause that is being audited?
• Is the evidence sufficient – is it sufficiently addressing the requirements enabling a fair and reasonable judgement about compliance?
• Is the evidence authentic? – is it genuine, real, and related to actual practice?
• Is the evidence current - is it relevant to the operations of the RTO at the time of the audit and reflects current industry/regulatory requirements?

Flexible

RTOs vary in size and scope, from a one-person provider delivering units of competency in a niche market in one location, to a large provider with numerous qualifications on its registration offered state-wide. The diversity of providers includes private providers, community providers, enterprise-based RTOs, industry-based RTOs, TAFEs, and schools.

 This diversity means that there is no ‘one size fits all’ approach to evidence of compliance or audit processes. The auditor must be open to considering the different forms of evidence to support making a judgement about compliance with the Standards and clauses. 

Fair open and transparent

The area being audited is informed about the audit in advance and given an opportunity to provide evidence of compliance in a form suited to the RTOs operations. The area being audited should be advised who they can contact if they have any questions or concerns before the audit including any conflict of interest with the auditor.

For site audits, the audit schedule will be communicated between the area being audited and the auditor.  The process is confirmed at the entry meeting. Auditors will provide information about their role when interviewing staff or clients. The auditor will also explain how the information participants provide will contribute to judgements about compliance with the Standards for RTOs. 

The Internal Audit Process

The internal audit process has 5 main steps, shown below:

1. Planning the Audit;
2. Resourcing – Team and Tools;
3. Conducting the Audit;
4. Reporting the Audit; and
5. Follow-up and Improvement.

1. Planning the Audit

Planning an audit is essential to ensure the audit is conducted in a systematic, fair, open, and transparent manner.  The VET sector uses a risk-based approach, all regulators in the VET system have a risk strategy available from their websites and RTOs should also use a risk assessment approach to decide where to focus their internal audit efforts.

Risk factors to consider include:

• systemic risks; and
• RTO risks.

Systemic Risks

Systemic risk is risk affecting a group of RTOs, specific industry areas or Standard, or the VET sector as a whole.  The impact of this type of risk is often significant and far-reaching and could potentially be detrimental to training and employment outcomes for graduates and on the reputation of VET. Factors that may determine systemic risk include:

• changes to assessment arrangements for training products;
• impacts of state and national VET policy and program changes;
• changes in the labour market and their impact on VET delivery; and
• trends of non-compliance with RTO Standards, training products or industry sectors. 

RTO Risks

RTO risk is risk that relates to an individual RTO.  It is often the result of decision and actions an RTO takes that if left unmanaged, could impact negatively on training outcomes for students. Risk factors may include:

• overall history of compliance, including areas of re-occurring non-compliance;
• nature and frequency of verified complaints;
• delivery of specific high risk training products;
• modes of delivery, including online, VET in Schools or apprenticeship programs;
• high volume or low volume delivery;
• delivery involving third party arrangements; and
• changes in training and assessment staff and other significant changes in the RTO operations. 

RTO’s can use their understanding of risks to determine what to include in the internal audit and how to conduct the audit. RTOs can gather information from a range of sources to inform their risk assessment. This could include:  

• complaints;
• engagement with industry;
• training product risk assessment;
• regulatory activities;
• enrolments and completions; and
• staff and client feedback.

Setting the objectives for your audit

Once you have conducted a risk assessment you will be able to decide what to audit (what clauses and training products to include), how and when to audit, and who will need to be involved in the audit (internal staff and external clients, such as students and employers).  This will set your RTOs internal audit schedule and help you determine what resources you need to conduct the audits.

2. Resourcing Internal Audits

RTOs differ significantly in terms of their size, scope, and location. Resourcing internal audits therefore poses a different challenge for every RTO.

The Internal Audit Team

An internal audit can be carried out by a single auditor or an audit team, consisting of members from within or outside the organisation. The composition of the audit team may differ from one audit to another.

The roles and responsibilities outlined below pertain to when an audit team is being used during an internal site audit. However, it is crucial to recognise that it is acceptable and feasible for a single person to conduct the audit, tailoring the roles and responsibilities to accommodate the size and scope of the RTO to ensure a manageable and effective audit process.

The Auditor / Lead Auditor (Team Audits)

• confirm the scope of the audit;
• contact the auditees and make an appointment for audit;
• identify and confirm resources (including audit team members and audit documentation) required to conduct the audit;
• review documentation and develop a plan and schedule for the audit in conjunction with the auditees and then confirm these arrangements;
• brief the audit team (if applicable);
• conduct the entry meeting;
• manage audit team resources by ensuring that there is effective communication between the members of the audit team, and by working with the auditee to ensure that the audit team have access to the materials, sites, and personnel they require;
• coordinate the audit findings by meeting with the audit team to discuss all the evidence collected; and
• conduct the feedback session with the auditee and confirm any follow-up.

A Co-auditor (Team Audits)

• participate in the entry meeting;
• identify and gather information;
• analyse information;
• evaluate information;
• report findings to the lead auditor;
• participate in the feedback session; and
• undertake other duties as requested by the lead auditor.

Irrespective of the make-up of the audit team, auditors should abide by a code of conduct or practice.  

It is also very important to acknowledge that when the auditor is internal to the organisation, there are some unique challenges that will be faced.  It is much harder to be dispassionate and objective when the auditor is internal to the organisation. They can ‘know too much’ before and after the audit. This may impact their ability to be fair and manage the conflicts of interest that almost inevitably occur.  Keeping confidences and being able to challenge or question the organisations traditions and way of working may also be difficult.   With this in mind, where possible the organisation should choose an internal auditor who is likely to be able to stay objective and keep confidences.  

Consultants

RTOs often hire consultants to aid in the establishment or upkeep of their compliance systems. When utilising a consultant, it's crucial that, RTO staff also clearly comprehend the systems, processes and practices. During audits, it's advisable not to involve the consultant, as this ensures that the auditor is assured of the RTO's ability to fulfil the requirements outlined in the Standards for RTOs.

Audit Tools

All auditors will use an audit tool of some form. The tool most commonly takes the form of a checklist that the auditor will use to record:

• evidence that has been provided during the audit;
• records of conversations from interviews;
• comments about the evidence that has been provided;
• any findings related to the clauses being audited; and
• any actions required by the auditee.

The form of the audit tool can vary significantly between RTOs and may be paper-based or electronic.  The key is that the auditor can use the audit tool to record the audit and findings thoroughly and accurately.

3. Conducting the Audit

All RTOs are structured and operate differently, focus on the needs of a range of client groups, have a unique scope of registration, and diverse business practices. In large RTOs, these differences can exist within the many business units or areas within the RTO. The important thing for an internal auditor is to remember to consider the effectiveness of the RTOs approaches in meeting customer and regulatory requirements. The auditor must have an open mind.

Opening Meetings

Opening meetings or entry meetings provide everyone involved in the audit an opportunity to understand the audit process, confirm arrangements and for the auditor to understand the way the auditees operate.  This last point is particularly important for larger RTOs where internal auditors may not be familiar with the operations of the area they are auditing.

The auditor is responsible for conducting the entry meeting which should include:

• introduction of the audit team (including observer, if applicable) and roles of team members;
• details of the scope of the audit;
• information on the audit schedule and structure;
• the confidentiality of all information obtained about the business during the audit;

Auditees should be invited to discuss their operations which could include:

• number and profile of learners;
• the range of clients and how relationships are managed;
• how training and assessment are planned and implemented;
• how industry is involved in ensuring that training and assessment are of a high standard;
• the number of staff and their roles and responsibilities;
• how consistency in the quality of operations is ensured; and
• how communication takes place within the area.

The opening meeting is also often the first opportunity the auditor has to begin building rapport with the auditees. This is important as audits are far more effective when everyone is comfortable to be open and communicate transparently throughout the audit process.

Communication Skills

When you conduct an audit, you may feel that your success is based upon your technical skills, your subject matter knowledge, your ability to be a professional sceptic and find out what’s really happening in an organisation, and you would be right, all of these are important. They all however pale into insignificance when compared to one skill, your ability to communicate.

Communication skills allow you as an auditor to establish a rapport with the auditee, establish trust, be open and transparent and achieve a genuine two-way exchange of information about an auditee’s position. It allows you to clearly convey thoughts, ideas and suggestions during meetings, interviews, and negotiations with auditees and management.  It means the audit will be conducted in a positive way and in a safe environment. 

Evaluating evidence and making an audit decision

Evidence that is collected must be evaluated to determine if it demonstrates that the organisation is meeting requirements. Auditors must determine:

• whether the RTO meets the requirements of each Standard;
• whether the results were achieved through a planned and systematic deployment of specific actions taken by the RTO, what gaps there are, if any, in the evidence and the causes of any poor outcomes achieved by the RTO;
• whether actions taken by the RTO are improving the way the RTO provides training, assessment, and client services; and
• whether the outcomes are sustainable throughout the registration period.

.When reviewing evidence, the following needs to be considered:

• relevance to quality outcomes, the Standards and clauses;
• alignment with other evidence collected for verification;
• sufficiency – perhaps a broader sample is needed;
• links to and inter-relationship between other Standards and clauses; and
• effective use of audit time and resources.

The evaluation of the evidence will determine whether the RTO is complying with the Standards. The results will be reported as:

• Compliance, which encompasses Good Practice and Opportunity for Improvement; or
• Non-compliance.

Closing meetings

The exit interview or closing meeting is your opportunity as the auditor to finalise the site visit. The closing meeting should not be rushed and should include:

• thanking staff for their assistance;
• outlining the strengths;
• discussing any opportunities for improvements;
• outlining the audit findings, including an overview of any non-compliance;
• allowing the auditee an opportunity to ask questions; and
• detailing post-audit process and timelines for audit completion.

4. Reporting Findings

An audit report is an important record used by many people in an organisation. The report should be clear, consistent, and accurate. The report must be:

• factual and only based on the evidence provided during the audit, or observed by the auditor;
• written in a way that enables the reader to understand what the audit outcome is for each Standard and clause and in cases of non-compliance, where rectification is required; and
• written in plain English using appropriate language and technical references.

Internal audits provide you with an opportunity to structure a report in a way that suits your organisation so consider what might work for you  organisation. Whatever form it takes, make sure it is used and useful. 

Example 1

You work in an organisation that has a Board. The Board wants a short, sharp overview of the audit findings to keep informed. However, the teams that are implementing actions need a more detailed report to guide their work.  The auditors therefore provide two different reports.

Example 2

The RTO is a small organisation and does not see value in a formal audit report. Instead, what is useful for the RTO is to develop an action plan as a team. This enables the RTO to focus on improvements.

When an auditor reports the findings of an audit there are four things that can be reported.

1. Compliance
2. Compliance with an opportunity for improvement
3. Compliance with good practice
4. Non-compliance.

When reporting compliance, the auditor states compliance has been demonstrated. It is the simplest outcome to report.

An opportunity for improvement means that the RTO is compliant, and the auditor is suggesting where improvements can be made. It is important to remember that an opportunity for improvement is a suggestion and the RTO can choose to adopt...or not.

Good practice is when the RTO has put systems or processes in place that are achieving outstanding results. When reporting good practice, the auditor needs to explain what the processes or actions are that are driving the results and what the impact is of the actions of the RTO. The auditor needs to clearly explain why the practice is outstanding.

If an auditor determines that an RTO is not complying with the Standards for RTOs they must be able to clearly articulate the gap. The framework below is a straightforward way to structure writing a non-compliance.

1. What is the requirement of the Standard – re-state the clause in simple terms.
2. What evidence did you see – this is where you would list/explain the evidence, but is not the place to make any judgement statements.
3. What is the gap between the evidence and the requirement of the Standard? This is where you explain the gaps, so that everyone can understand the gap and its impact.
4. What do you want the RTO to do about the non-compliance? This is an opportunity that is unique to internal audits, you can provide very specific advice about how to rectify non-compliance.

Audit reports must be provided in a timeframe that is meaningful for the RTO. If the report is delayed, often the impetus created by the audit process is lost. In most circumstances, reports are provided within 1-2 weeks after the closing meeting.

5. Follow up and Improvement

After an audit, the auditee needs to consider the audit findings and determine what actions will be taken. In most cases, the decision about the actions will be a team activity. The benefit of this is that the planning can use collective wisdom to determine the best way to address a non-compliance. Team planning enables the RTO to consider the capacity and capability of the team to prioritise actions and determine time frames for completion. Having the team involved in the planning of any corrective actions increases ownership and the likelihood that the auditees will adopt and implement the changes.

The actions taken by the auditees should then be reviewed to determine their effectiveness in addressing the non-compliances. If the actions are not as effective as intended, the auditees can then consider what did not work as intended, and what else needs to change to correct the non-compliance.

Closing out an audit

An audit can be closed out or finalised when all the actions have been completed and are effective.

The report and completed action plans are important contextual records for the RTO and as a result need to be stored within the RTOs record system.

Closing out the audit is a point where the RTO should review the audit schedule. This would include considering if anything needs to be included or changed about the schedule. If the actions from the audit resulted in the development of new systems or processes, it is prudent to check that they have been implemented properly and are working by including the area in future audits.

Focusing on Continuous Improvement

Every evaluative activity in an organisation provides an opportunity to learn and improve. Focusing on improvement is a practice and mind-set where the organisation is always looking for better ways to do things. This practice leads to the development of products, services, workflows, and other aspects of the organisation so they become more effective and efficient over time.

There are many models for continuous improvement that RTOs can use, but all ask:

• what did we learn in this situation? and
• what can we do better?

Remember that you can be asking these questions before, during and after the internal audit process by building reflective practice into the RTOs daily work. While it sounds very clichéd, quality is not a destination, it really is an ongoing journey. Internal audits are a great way to gather data and inform the quality journey in your organisation.