Session rating system

05 Feb, '21
Ron Malinowski WX4GPS

This is a great Idea, it will make VEC adhere to high standards and this will help promote our hobby is a professional way.

05 Feb, '21
Herb Weiner

Great idea. Let candidates choose sessions with high ratings.

05 Feb, '21
Steven

Great idea!!!

05 Feb, '21
Darrell Epperson

There are too many sessions doing questionable things when testing. We hear about it at some of our sessions.

05 Feb, '21
Mike Rogers

Good idea and a selection tool to pick a test session.

05 Feb, '21
Roger McEntarfer

Great Idea.

05 Feb, '21
Ron McCall

I agree. If people can see that there are viable choices, they may very well go with folks that have built a reputation for caring about what they do, and how people perceive their helpfulness. While no one can kibitz and coach people on completing their test, they can be very encouraging and make examinees want to come back for upgrades.

05 Feb, '21
Dennis K3DCK

All for it!

05 Feb, '21
Richard Bateman

If someone would like to make some mockups of a better exam session listing / schedule page I'll certainly consider it -- and if it has a rating system I'll consider adding that as well.

05 Feb, '21
Bryan Gonderinger

I'm going to be contrarian here. With no other information, a 1-5 rating is not too useful, in my opinion. I assume this "rating" would be per session callsign. In our case, our group uses a single callsign for all sessions (the "club callsign"), but the in-person sessions are run by multiple separate groups of VEs. This proposal would not distinguish them.
Perhaps people wind up rating ARRL sessions lower than Laurel sessions (when they've used each) because the ARRL sessions charge a test fee, whereas Laurel doesn't.

What would perhaps be more useful overall would be an after-test survey/comment form, the results of which would be available to the owners (and possibly co-owners) to receive useful feedback and address in their own sessions.

If someone has a problem with a session I'm running, having the overall callsign rating drop by 0.2 isn't going to tell me anything.

05 Feb, '21
Allan Batteiger

I will see if Jason and I can come up with something. I would like the listings to default to only sessions with openings first. I have had a number of comments recently about the number of sessions listed that had no slots available. Why list them at all if they are full. In reference to Bryan's comment The question was of test experience. You could have several questions in the rating, but the Question I was asking was overall test experience. If you team is properly trained they should be making the applicant feel welcome, and their time ( the applicants) is important, They should be able to put the applicant at ease in taking the test online. They should be able to help the applicant address any technical issues that affect their test experience. They should hope that at then end of the test the applicant would want to some back for more tests ( upgrade or restest).

05 Feb, '21
Chris Flinn

I agree with the default being sessions with openings first. That does open a door that has been exploited and continues to be done daily with a certain VE team that lists their sessions at the earliest each day so they show up first. I think the filter should be set to show sessions that have a couple people in them first and then the wide open ones towards the bottom. Have it randomize how it selects the amount needed to be above the sessions with nobody listed that way it cant be exploited again by always having xx amount of people signed up.

05 Feb, '21
n2ygk

@Chris I don't think we should try to game the listing system by requiring sessions sorted only after someone signs up. See https://features.examtools.org/suggestions/117046/add-show-only-available-checkbox-on-hamstudyorgsessions

06 Feb, '21
Allan Batteiger

I think the current list by day is fine but only show days / sessions with slots open at least for online sessions. That should prevent gaming the listing. I am sure some team will come up with a way to game the listing and that will have to be addressed as needed.

07 Feb, '21
Bryan Gonderinger

What are the concerns about "gaming the system?"

I was of the opinion that we all have the same goal here - to allow as many people who want to take ham licensing exams to be able to do so. Whether they do this with my team, your team, or some other team doesn't make a difference.

I'm not aware of any competition going that would pit one VE team against another, but I could be missing something.

I suppose at some point, if we have too many VE teams offering sessions and wind up with 50 sessions a day, each with just one candidate, that this would perhaps be a problem where we would want to focus on filling available sessions first, but I'm not sure if we are there yet.

If we are going to implemetcategorization/sorting of the list, being a cheap ham, my vote would be to sort on test cost first.

07 Feb, '21
Nick n1cck

Bryan -
We do of course all have the same goal of testing people. Of course, people being people, VE teams and VECs all look at stats as “not a contest” - just bragging rights. ;)

I think most people’s concern with gaming the system doesn’t actually have anything to do with stats, numbers, or competition, however. What we have seen historically (dating back to April, 2020) is that some teams will hoard candidates on a waiting list and/or schedule going out months; however, other teams will have open slots and scheduling sooner. The concern is that if any given team is actively making it so that their sessions show up first in the listing but they don’t actually have availability at the time they claim, that’s actually a negative experience for the candidate who could have been better served by people simply posting their actual schedules.

Cost games the system inherently, because VECs set global prices (as required by P97). Laurel would always be first, W5YI and ARRL never

08 Feb, '21
n2ygk

See my suggestion just added to the topic of listing unlisted sessions at https://features.examtools.org/suggestions/117046/add-show-only-available-checkbox-on-hamstudyorgsessions. Comments here are off-topic as this topic is only about ratings.

And we are not competing in a negative way, just trying to make sure candidates have the best possible experience. We are after all trying to enable new people to join or upgrade their skills for our avocation.

Comments: 17