Gregory Abowd from Georgia Tech is giving the tutorial on Evaluation of Real Deployments in Ubicomp. He is talking about what are the issues you need to think about when deploying a ubicomp system. Very often papers get rejected because there is no evaluation of the system in a deployed environment. What is important is how to do research with human subjects, how there needs to be an ethics approval, compensation, getting the right demographic and retaining participants. Sometimes it is OK to be the end-user and the designer. We also have to ask users what they think through surveys, questionnaires, and interviews. You need to spend a lot of time to prepare the survey or interview, but in the end the analysis is much easier, so basically you should not rush writing a survey or questionnaire.
In designing a study, you need to be a devil's advocate and ask yourself what things could affect the interpretation of the study and controlling variables. What kind of study is it, within subjects design or between subjects design. How do we get information from a person's every day life? We don't want to follow them every hour, every minute, by instrumenting a camera or recording device. Instead, the better way is to use experience sampling, where you interrupt the person to gather relevant information.
With experience sampling, you can use a diary where you ask the participant to record what they were doing, where they were doing it, why they were doing it. However, there are social-legal implications of recording this. Basically, in the end, we need to balance tracking participants with asking participants questions so that can be able to match between them.
No comments:
Post a Comment