The project's final report and the prototype demonstrator are now available:
Report on this website
Demonstrator at: http://research.ucas.com/efifa/.
Monday 3 August 2009
Wednesday 18 March 2009
Short session on the demonstrator at UCAS' state school Service Advisory Group, 18 March 2009
The audience (about 8 schools representatives) had been briefed on EFIFA at their previous meeting, and this was an opportunity to show them the demonstrator and get reactions and comments. Note that the version of the demonstrator didn’t have the ‘student view’ section. The audience also had the opportunity to fill in response forms and to take them away, have a look again later and send responses by email.
There was a generally favourable reaction to the demonstrator. They thought it was very easy to use – and therefore wanted HEIs to provide feedback properly! They believed that it would be a very good method of giving feedback.
Specific comments were few:
(1) Reasons for rejection should be recorded as a matter of routine, and good practice, when a decision is made, and therefore with an electronic system to support this, there should be no reason why blanket feedback for unsuccessful applicants couldn’t be given.
(2) It was noted that almost all feedback now is given when the decisions are made. This type of system should support that approach.
(3) The ability for applicants to request feedback via UCAS Track was seen as important, and it might stimulate HEIs to put in place systems to satisfy the demand.
(4) The ability to give free text individualised feedback was seen as important.
There was a generally favourable reaction to the demonstrator. They thought it was very easy to use – and therefore wanted HEIs to provide feedback properly! They believed that it would be a very good method of giving feedback.
Specific comments were few:
(1) Reasons for rejection should be recorded as a matter of routine, and good practice, when a decision is made, and therefore with an electronic system to support this, there should be no reason why blanket feedback for unsuccessful applicants couldn’t be given.
(2) It was noted that almost all feedback now is given when the decisions are made. This type of system should support that approach.
(3) The ability for applicants to request feedback via UCAS Track was seen as important, and it might stimulate HEIs to put in place systems to satisfy the demand.
(4) The ability to give free text individualised feedback was seen as important.
Saturday 24 January 2009
Highlight Report Summary - 24 January 2009
User requirements stage was completed and a functional specification for the demonstrator produced and circulated within the team. We confirmed Matthew Willard as our developer for the demonstrator; he will work closely with Neil England (Business Analyst) and with Dave Morgan (Senior Supplier) to ensure that we meet our targets.
A further team workshop was held to confirm that the functional specification correctly encapsulated the requirements.
Neil England is collecting evidence from the usage of the existing system for presentation to the project team.
Development of the prototype has started.
A further team workshop was held to confirm that the functional specification correctly encapsulated the requirements.
Neil England is collecting evidence from the usage of the existing system for presentation to the project team.
Development of the prototype has started.
Wednesday 3 December 2008
Highlight Report Summary - 2 December 2008
Collection of user needs and analysis of them continued throughout the period. As relatively few applicant replies had been received from the first trawl, a second trawl has been undertaken and this has more than doubled the number of responses to over 200. This extra information will be fed into the analysis and modelling processes. In addition engagement of the required number of HEIs has not been completed, but is being followed up rapidly, so that information can be included in the project before Christmas. We have not been able, to date, to follow up the use of the current UCAS-based system; it is likely that significant information in this area will not be available within the time scales of this project.
A technical model of the existing UCAS feedback system has been produced for comparison with our proposed new one.
We stimulated applicant replies by including a random draw for an iPod Nano, which worked well.
A list of requirements for the new system has been drawn up and was discussed at a team workshop. We are now in a position to commence work on the revised process (Stage 3 of the project).
NOTE: During Sarah Davies’ maternity leave (from 31 Nov 08) Paul Bailey will be our new JISC contact.
A technical model of the existing UCAS feedback system has been produced for comparison with our proposed new one.
We stimulated applicant replies by including a random draw for an iPod Nano, which worked well.
A list of requirements for the new system has been drawn up and was discussed at a team workshop. We are now in a position to commence work on the revised process (Stage 3 of the project).
NOTE: During Sarah Davies’ maternity leave (from 31 Nov 08) Paul Bailey will be our new JISC contact.
Tuesday 25 November 2008
More questions, considerations and assumptions
16. What HEI Feedback rules are we going to display in the prototype, e.g. one HEI may agree to provide Feedback within 28 days and another may agree to provide Feedback within 20 working days …… similar, but different ?
17. Is it worth considering a 'limited' set of responses, which meet the needs of those institutions who elect to join the pilot scheme and, at the same time, experiment with the idea of it becoming a global, but comprehensive, list of potential feedback replies ?
N.B.1. This may eliminate the need for the prototype to provide the facility for Code and associated text maintenance by the institutions.
N.B.2. Don't over-simplify, but don't invite litigation.
N.B.3. This approach could be very positive for institutions that have courses that are, largely, over-subscribed.
18. From the "Applicant Feedback Study Final Report / 29 March 2007", which asked employers about feedback, the most significant item upon which feedback was given was "Performance at interview". However, the sample feedback letter in the Schwartz report (14 September 2004) makes particular reference to the applicants' personal statements. Many institutions don't, or only rarely, interview applicants, so the personal statement assumes the same significance as the employer interview for an HEI applicant and this should be made very clear in the Entry Profile, possibly with examples of what the institution is looking for.
N.B. This is not so much a function of the prototype as the operation within the institutions. However the production of the Letter/Email is a function of the prototype.
19. HEs, FEs and Applicants appear to agree, from the survey results that 'standard' paragraphs combined with more specific free text provide the best form of Feedback.
20. How do we ensure the prototype is expandable to be used by FE admissions, Post Graduate admissions and employers providing feedback to job applicants ?
17. Is it worth considering a 'limited' set of responses, which meet the needs of those institutions who elect to join the pilot scheme and, at the same time, experiment with the idea of it becoming a global, but comprehensive, list of potential feedback replies ?
N.B.1. This may eliminate the need for the prototype to provide the facility for Code and associated text maintenance by the institutions.
N.B.2. Don't over-simplify, but don't invite litigation.
N.B.3. This approach could be very positive for institutions that have courses that are, largely, over-subscribed.
18. From the "Applicant Feedback Study Final Report / 29 March 2007", which asked employers about feedback, the most significant item upon which feedback was given was "Performance at interview". However, the sample feedback letter in the Schwartz report (14 September 2004) makes particular reference to the applicants' personal statements. Many institutions don't, or only rarely, interview applicants, so the personal statement assumes the same significance as the employer interview for an HEI applicant and this should be made very clear in the Entry Profile, possibly with examples of what the institution is looking for.
N.B. This is not so much a function of the prototype as the operation within the institutions. However the production of the Letter/Email is a function of the prototype.
19. HEs, FEs and Applicants appear to agree, from the survey results that 'standard' paragraphs combined with more specific free text provide the best form of Feedback.
20. How do we ensure the prototype is expandable to be used by FE admissions, Post Graduate admissions and employers providing feedback to job applicants ?
Questions, considerations, assumptions
We came up with these questions, considerations and assumptions, for discussion:
1. The prototype will be aimed at Feedback to Applicants who have been rejected by institutions.
N.B. Should Feedback automatically form part of the selection process, no matter whether the Application be rejected or the Applicant made an Offer ?
2. Needs to support:
- Code and associated text maintenance by the institutions
- Feedback to the Applicant by the institution
- The ability to provide Feedback via 'Free Text', as well as calling up codes.
3. A key objective of the project is to implement something which will have insignificant impact on the institutions; in terms of time and money spent.
4. Codes and associated text are most likely to come from the institutions and, possibly, faculties within the institutions, rather than applying a national standard set.
N.B. The responses from an Art & Design institution may be very different from those from a Science oriented institution. A set of nationally agreed responses will, almost certainly, be too broad and of little real use to the Applicants.
5. It should be possible to provide Feedback to an Applicant, who has requested it, at any point in their Application process, i.e. not restricted to Choices where the Application has not moved on to another institution.
6. Would it be 'useful' for an Applicant to be able to request Feedback via UCAS's systems ?
N.B. A risk associated with this is that if made easy (and why not ?) it will encourage Applicants to always request Feedback, which may overburden the HEIs.
7. The requirement is for a Demonstrator, hence, is there a need to support UCAS, CUKAS, GTTR and UKPASS institutions ? Suggest just UCAS. However, will this satisfy the institutions broadly enough ?
8. How will the prototype Feedback be made visible to the Applicant ?
9. Will the Applicant still receive an Email or Letter ?
10. How will the prototype Feedback combine with that already available in Apply09 ?
11. Will the prototype transactions instantly update TOPAZ ?
12. Will a CSU operator be able to see the results of what happens in the prototype ?
13. Will the prototype replicate TRACK ?
N.B. This will not be necessary if TOPAZ is updated.
14. How far 'down the pipe' do we need to go with the prototype, e.g. do we want/need to accommodate 'Extra' Applications ?
15. Bearing in mind that Route 'B' is disappearing, in its current form, does it need to be catered for in the prototype ?
N.B. GTTR will remain sequential.
1. The prototype will be aimed at Feedback to Applicants who have been rejected by institutions.
N.B. Should Feedback automatically form part of the selection process, no matter whether the Application be rejected or the Applicant made an Offer ?
2. Needs to support:
- Code and associated text maintenance by the institutions
- Feedback to the Applicant by the institution
- The ability to provide Feedback via 'Free Text', as well as calling up codes.
3. A key objective of the project is to implement something which will have insignificant impact on the institutions; in terms of time and money spent.
4. Codes and associated text are most likely to come from the institutions and, possibly, faculties within the institutions, rather than applying a national standard set.
N.B. The responses from an Art & Design institution may be very different from those from a Science oriented institution. A set of nationally agreed responses will, almost certainly, be too broad and of little real use to the Applicants.
5. It should be possible to provide Feedback to an Applicant, who has requested it, at any point in their Application process, i.e. not restricted to Choices where the Application has not moved on to another institution.
6. Would it be 'useful' for an Applicant to be able to request Feedback via UCAS's systems ?
N.B. A risk associated with this is that if made easy (and why not ?) it will encourage Applicants to always request Feedback, which may overburden the HEIs.
7. The requirement is for a Demonstrator, hence, is there a need to support UCAS, CUKAS, GTTR and UKPASS institutions ? Suggest just UCAS. However, will this satisfy the institutions broadly enough ?
8. How will the prototype Feedback be made visible to the Applicant ?
9. Will the Applicant still receive an Email or Letter ?
10. How will the prototype Feedback combine with that already available in Apply09 ?
11. Will the prototype transactions instantly update TOPAZ ?
12. Will a CSU operator be able to see the results of what happens in the prototype ?
13. Will the prototype replicate TRACK ?
N.B. This will not be necessary if TOPAZ is updated.
14. How far 'down the pipe' do we need to go with the prototype, e.g. do we want/need to accommodate 'Extra' Applications ?
15. Bearing in mind that Route 'B' is disappearing, in its current form, does it need to be catered for in the prototype ?
N.B. GTTR will remain sequential.
Monday 3 November 2008
Highlight Report Summary - 3 November 2008
The User Requirements Stage was due to finish at the end of October. Although we have received a number of replies to the surveys, it would be useful to receive more, so we propose to extend the User Requirements Stage until the middle of November. This will also allow more time for the engagement of developer staff, although it will also have an impact on the Prototype Development schedule (see below).
The project is collecting requirements from UCAS’ schools Service Advisory Group meetings in November.
As noted in the previous highlight report, collection of HEI staff experience of the existing system for 2009 entry will happen in the second half of October and early November, via existing surveys and our consultation contacts.
Neil England has completed the initial modelling work.
The project website and project blog were published a few days ago, which was approximately 7 weeks later than initially scheduled, owing to an unexpected requirement for inclusion in UCAS’ internal test and release programme.
The project is collecting requirements from UCAS’ schools Service Advisory Group meetings in November.
As noted in the previous highlight report, collection of HEI staff experience of the existing system for 2009 entry will happen in the second half of October and early November, via existing surveys and our consultation contacts.
Neil England has completed the initial modelling work.
The project website and project blog were published a few days ago, which was approximately 7 weeks later than initially scheduled, owing to an unexpected requirement for inclusion in UCAS’ internal test and release programme.
Subscribe to:
Posts (Atom)