NAP news, September 2016

NAPLAN Online passes its first test

ACARA and its partners have undertaken trialling in preparation for the move to NAPLAN online. The results have indicated that NAPLAN Online is on track to be implemented from next year and deliver better assessment, more precise results and a faster turnaround of information to those schools going online. 

“We are leaving nothing to chance to ensure we are ready to begin the move to NAPLAN online from next year,” says ACARA CEO, Robert Randall, “this is why significant planning, development, research and trialling are going on to make sure we are all ready to move online.”

Initial results from the trials and studies undertaken this year have been mostly positive. The platform trial and readiness test (PTRT) confirmed that the majority of schools have sufficient capacity to access the platform and administer the online assessment. As expected, a number of technical issues were identified, largely relating to the locked-down browser, local filtering and proxy settings, and wireless access and connectivity issues. Jurisdictions will undertake further analysis to ensure readiness of schools in the lead-up to May 2017.

Platform trial and readiness test (PTRT)

During August 2016, students from more than 1,000 selected schools nationally took part in a trial of the new assessment platform for NAPLAN Online. 

Students took one or two online readiness tests, which included examples of the full range of NAPLAN online questions and a separate writing test. 

Results received from the trial have been positive:  

• The platform performed well and the trial confirmed the majority of participating schools could undertake the online assessment successfully and with minimal issues.
• Students engaged with the online format positively, were focused and said they enjoyed the experience of NAPLAN Online.
• Of those students who participated in a feedback survey
• more than 86 per cent liked using a computer for the writing assessment.
• more than 83 per cent liked using a computer for all other domains (not all students did both the writing and the combined tests). 

About the item trials and studies 

Item trials and studies carried out by ACARA have provided information that will help develop the NAPLAN Online 2017 test. The online item trial evaluated test items (questions) in a sample of around 180 schools across Australia.

The scaling study was delivered in around 260 schools in August and September. The scaling study has collected information about the performance of students in the multistage test design (or ‘tailored tests’) to be used in NAPLAN Online. This will guide and support the development of the tests.

The linking study, which will continue until October 2016, is being conducted to ensure that paper-based tests can be compared with the NAPLAN online tests. On completion, around 180 schools will have participated in this study.

Feedback from schools has been mainly positive. Students found the new online tests engaging; test administrators found the platform easy to operate. Initial results from the study demonstrate that items and tests are performing as expected. Data analysis is now underway.

“The majority of students were extremely positive about the online testing experience.”
State school deputy principal, Queensland

“Students engaged incredibly well, were focussed and later said that they really enjoyed the experience.”
State school deputy principal, Queensland

“All in all, we had a very productive and informative day – certainly a great start to our NAPLAN online experience!”
State school acting deputy principal, Queensland

Feedback from schools will be used to identify improvements required before the first online NAPLAN assessment in May 2017. 

ACARA responds to APPA's NAPLAN online trial survey

ACARA welcomes the results of the Australian Primary Principals Association (APPA) NAPLAN online trial survey. The survey sought feedback from principals of schools participating in the recent platform trial and readiness test (PTRT) of the national online assessment platform.

The APPA survey results confirm the generally positive outcomes of PTRT testing nationally. The concerns expressed by APPA in response to the survey, however, do not reflect the experience of PTRT for most schools or the feedback and results from APPA's own survey.

APPA correctly noted in its survey findings that the intention of PTRT was to ensure any ‘challenges faced by schools to move to an online testing environment are identified and addressed’.

The implementation planning for NAPLAN online provides for issues arising from PTRT and feedback from schools and test administration authorities to be addressed before moving online from 2017.  Further improvements and enhancements are being made to the assessment platform. In in the lead-up to May 2017, states and territories are undertaking additional analysis and work to address logistical and administrative issues, which occurred during PTRT, and to ensure readiness for those schools moving online. 

The platform trial and readiness test (PTRT) for NAPLAN online

Federal, state and territory ministers agreed the National Assessment Program – Literacy and Numeracy (NAPLAN) will move online from 2017 over a three-year period. The goal is for around 10 per cent of Australia schools to be conducting NAPLAN online in 2017, with more schools in 2018 and for all schools to conduct tests online by 2019.

The platform trial and readiness test (PTRT) was undertaken in August 2016. It involved over 80,000 students and more than 1,000 selected schools in seven states and territories. Over 160,000 trial tests were completed. The majority of the schools that took part in this PTRT are expected to participate in NAPLAN online in May 2017.

PTRT was designed to be a ‘practice run’ to test the online assessment platform as well as to give schools an opportunity to assess their readiness, including schools’ ICT infrastructure, test administration processes and classroom logistics. The test was held over a nine-day period, which replicated the May 2017 NAPLAN online testing window.

Results received from the trial have been positive: 

• The platform performed well and the trial confirmed the majority of schools that participated can successfully undertake the online assessment with minimal issues.
• Students engaged with the online format positively, were focused and said they enjoyed the experience of NAPLAN online.
• Of those students who participated in a feedback survey
  • more than 86 per cent liked using a computer for the writing assessment
  • more than 83 per cent liked using a computer for all other domains (not all students did both the writing and the combined tests).

The platform performed well and the trial confirmed the majority of schools that participated can successfully undertake the online assessment with minimal issues.

As expected, a number of schools experienced some technical issues, related mainly to the locked down browser, proxy settings, connectivity or log-on issues. A few schools could not complete tests due to factors such as low bandwidth and computer freezes.

APPA survey

The results of the APPA survey confirmed the positive responses and survey results received by ACARA and states and territories. Positive results from the APPA survey include:

• 77 per cent of principals agreed or strongly agreed their schools experience was positive.
• 77 per cent of principals agreed their school was ready to undertake NAPLAN online.
• 89 per cent of principals agreed or strongly agreed their students followed the format and test instructions with ease.
• More than 80 per cent of principals agreed or strongly agreed the devices used by students successfully handled the test requirements.

Despite these positive findings, APPA has raised a number of concerns arising from PTRT. These included: concerns over technology requirements; a perceived need for additional trials and testing by schools, and additional training and support for school staff; the need for school systems to ensure equity of access to technology and devices.

Response to PTRT

APPA should be assured that significant planning, development, research and trialling have been undertaken and ACARA is continuing to address issues raised by the association and ensure all are ready to move online from 2017.

The APPA survey was completed by 191 principals (out of from more than 1,000 schools that undertook PTRT). This is a small sample only, it does not represent the experience of most schools that undertook PTRT. The majority of principals in the APPA survey reported their schools were ready to move to NAPLAN online next year – which is a great result in light of the fact that not every school is required to be ready for NAPLAN online from 2017 with only 10 per cent of schools anticipated to take part next year.

Technology issues and connections. Issues with technology and connections such as computer freezes, browser issues, difficulties logging on and additional time and workload required of schools in preparation and set up were to be expected in a trial.  PTRT involved new processes and procedures to test administration and classroom logistics.

The feedback from schools and jurisdictions has identified improvements to school procedures and classroom logistics as well as to the platform to enhance the user experience. Improvements will be made throughout the remainder of 2016 to ensure readiness to transition online from May 2017. Schools have enough time to address these issues in readiness for 2017, 2018 or 2019.

Additional testing and trials. Schools that took part in PTRT will stay connected to the online training environment and will be able to continue testing administration, logistics and technical readiness. The NAP website contains information for schools on minimum technical requirements to participate in NAPLAN online, providing a capacity-planning tool to help schools assess their technical readiness. This tool also provides suggestions on adjustments to meet the requirements.

From early October 2016, students, teachers, parents and the broader community will also be able to practise questions online through a public demonstration site of the new NAPLAN online assessment platform. This site will contain a mini-test with sample online questions, covering all domains for each NAPLAN year level. This site will be accessed via ACARA’s National Assessment Program (NAP) website.

For schools that cannot access the internet to the level required for online testing, alternative information technology solutions will also be available.

States and territories will support schools as they move online. The support may include training and development in addition to handbooks and technical user guides.

ICT skills of students.  It is important that schools ensure their students are familiar with the device they will be using for NAPLAN online. Using ICT should already be a part of the teaching and learning process for all schools as detailed in the Australian Curriculum, which recognises the importance of equipping students with the skills, knowledge and understanding of computing that they will need to participate effectively in the digital world. The development of these skills begins in the Foundation Year.

Testing window.  More than 86 per cent of principals in the APPA survey estimated NAPLAN online testing would take 3–10 days to complete.

The expectation of length of time to complete NAPLAN online is within the planned NAPLAN online testing window, which will be nine days compared to the current three days for the paper tests. This will allow schools to schedule tests according to the number of devices available and their bandwidth.

APPA also notes some concern regarding Year 3 students conducting the NAPLAN writing assessment online. Education ministers have sought further advice on this issue and have agreed that states and territories will make their own decisions whether Year 3 writing will be conducted online or on a pen-and-paper basis in 2017.

Feedback from schools trialling the NAPLAN online assessment platform

Students engaged incredibly well, were focused and later said that they really enjoyed the experience.
School Deputy Principal, Queensland

Though out testing isn't over ... I just wanted to tell you and your team how impressed I've been with the program you've put in place. The training, software and phone support have been most impressive and our testing has been mostly hassle-free.
Teacher, WA primary school