Appstoremining

2013-04-15T22:18:07Z

Maalej:

This page will contain data sets from the 2013 RE study "User Feedback in the AppStore: An Empirical Study" by Pagano and Maalej.

Appstoremining/

2013-04-15T22:17:17Z

Maalej: Created page with 'x'

BR/interviews

2013-01-29T10:20:31Z

Maalej: moved BR/interviews to Business rules interviews

Here you can download the Interview Forms [http://www.teamweaver.org/wiki/index.php/File:Interview_Template_BusinessRules.pdf] for the business rules study conducted at TCS. More public information will be available on this site from April/Mai 2013. Please also check the news on:

mobis.informatik.uni-hamburg.de

Next!

2012-03-16T17:03:50Z

Maalej:

== NEXT! - A Tool for Context-Aware Task Recommendations ==

by Vincenz Dölle & Walid Maalej

'''Material:'''
* [http://home.in.tum.de/~doelle/fse2012/ISE_IM-project-description.txt ISE_IM project description]
* [http://home.in.tum.de/~doelle/fse2012/ISE_IM-task-list.txt ISE_IM task list]

* [http://home.in.tum.de/~doelle/fse2012/ISE_POLL-project-description.txt ISE_POLL project description]
* [http://home.in.tum.de/~doelle/fse2012/ISE_POLL-task-list.txt ISE_POLL task list]

* [http://home.in.tum.de/~doelle/fse2012/Task-Comparison-Survey.pdf Task Comparison Survey] (build with [http://www.limesurvey.org LimeSurvey])

<br />
'''Additional Information:'''

''List of Students''

{|
| Student ID || Project
|-
| 1 || ISE_IM
|-
| 2 || ISE_IM
|-
| 3 || ISE_IM
|-
| 4 || ISE_POLL
|-
| 5 || ISE_POLL
|-
| 6 || ISE_POLL
|-
| 7 || ISE_POLL
|}

Woogle4MediaWiki/Instrumentation addon

2011-08-06T11:02:59Z

Happel: Created page with '== Purpose == The purpose of the instrumentation addon is to carry out (usability) experiments in MediaWiki. In a nutshell, different Wiki users receive different experimental tr…'

== Purpose ==
The purpose of the instrumentation addon is to carry out (usability) experiments in MediaWiki. In a nutshell, different Wiki users receive different experimental treatments (i.e., a different text or image) based on a certain hypothesis or design goal (i.e., improve clickthrough-rate). The addon logs their response/activity after receiving the treatment. Statistically analyzing the resulting data can help researchers or designers to judge their hypothesis resp. design goal (c.f. http://en.wikipedia.org/wiki/A/B_testing).

This addon provides a set of core features for this purpose (detailed description below):
* Configuring experimental groups & treatments
* Assignment of users to groups
* Instantiating experimental treatments at runtime
* Logging user activities
* Covering data protection and research ethics topics such as user pseudonymization & informed consent

=== Reusability ===
The instrumentation addon was designed as part of [http://www.mediawiki.org/wiki/Extension:Woogle4MediaWiki Extension:Woogle4MediaWiki] to help researching its usage.

However, its general mechanism can be easily adapted for other purposes/extensions as well.

Technically, one could therefore use the whole Woogle4MediaWiki extension and disable all its original features, to just use the instrumentation part. If useful for many users, one could also think about extracting the instrumentation functionality out of Woogle into an independent extension (interested developers should contact us).

== Features ==

=== Defining evaluation groups ===
The core of the instrumentation addon is the definition of the different experimental groups (see <code>instrumentationGroups</code> in the function woogleInstConfig in [https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/addons/Instrumentation/Instrumentation.php Instrumentation.php]). An '''arbitrary''' number of groups is possible. For evaluation purposes, the typical setup is to treat one ("control") group with a standard setting, and to treat further groups with modifications based on different experimental conditions. A simple example would be testing if a visual advertisement banner results in more clicks (compared to a standard text link). See also [http://en.wikipedia.org/wiki/A/B_testing A/B_testing].

The evaluation groups of the instrumentation addon allow for arbritrary modifications. In the context of Woogle, it makes use of Woogle's powerful configuration capabilities. Many options of the Woogle extension can be configured via [https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/includes/WoogleConfig.php WoogleConfig.php] and according user preferences.

The evaluation groups can be understood as pre-defined, grouped sets of user preferences which are assigned to different users. I.e., '''for each evaluation group defined in <code>instrumentationGroups</code>, all WoogleConfig.php-parameters can be set to a certain value for that group, resulting in a different user experience/different experimental treatments for each group'''.

While this is the current default setup, the instrumentation addon is only loosely couple to WoogleConfig.php. Arbitrary other ways of implementing different treatments are possible.

=== Assigning users to evaluation groups ===
The assignment of users to the configured evaluation groups (see above) takes place in the function assignInstrumentationGroup() (in [https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/addons/Instrumentation/Instrumentation.php Instrumentation.php]). Currently, only a ''random'' assignment is implemented, but the function can be easily modified.

For logged in users, the assignment will only be made once and remains stable across site visits.

=== Pseudonymization ===
For logging purposes, each user is assigned an anonymized id ([http://en.wikipedia.org/wiki/Pseudonymization psyeudonym]).

It is implemented in <code>WoogleUtil::getAnonUserID()</code> as follows:
* for anonymous users: their hashed IP address
* for logged in users their RandomUserID generated by Woogle or - if this is null - their hashed MediaWiki user name (RandomUserId = <code>md5(rand()</code>)

For logged in users, the pseudonym is stored in the MediaWiki database and will remain consistent unless the user resets it via user preferences. However, the idea si that the connection between user name and pseudonym is not visible for the evaluator - only the Wiki Sysop might make this connection.

=== Informed consent ===
Users can optionally be presented with a customizable information dialog which explains the goal of the study and requires them to explicitly decide on if to participate or not.

[http://en.wikipedia.org/wiki/Informed_consent#Research Informed consent] is only available for logged in users - not for anonymous accounts! It is realized by the [http://www.mediawiki.org/wiki/Manual:Special_pages Special Page] [https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/addons/Instrumentation/WoogleSpecialStudy.php WoogleSpecialStudy].

=== User preferences ===
User may also disable (or re-enable) their participation in [http://www.mediawiki.org/wiki/Preferences Preferences].

=== Logging user activities ===
By "logging targets" we mean user activities that should be logged by the instrumentation addon. In general, it can log arbitrary activities.

The usual workflow is like that:
# Think about user activity which should be logged
# Find a place in the MediaWiki/extension code which is suitable to determine if a user carries out an activity or not
# Add instrumentation logging code (i.e., call InstrumentationLogger; see below)
# Later: analyze logfiles written

Example:
# We want to log successful login activities
# The [http://www.mediawiki.org/wiki/Manual:Hooks/UserLoginComplete UserLoginComplete] seems to be a good place to determine user logins
# We register for this hook and call InstrumentationLogger (actual example [https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/addons/Instrumentation/Instrumentation.php in here]).

Due to their modular nature, [http://www.mediawiki.org/wiki/Manual:Hooks MediaWiki hooks] have turned out as a good choice to add logging code, but the instrumentation addon is not limited to this.

The instrumentation addon comes with a number of pre-defined logging targets (defined in [https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/addons/Instrumentation/Instrumentation.php Instrumentation.php], which can be easily extended:
* User logging in (code: user_login)
* User editing new page (redlink)
* User creating a watch (watch_created)
* User saving an article (article_save)
* User saves/changes preferences (pref_save)
* User receives search result with some results (mwsearch_results)
* User receives search result with no results (mwsearch_noresults)
* Woogle4MediaWiki-specific logging targets
** User executes Woogle search (woogle_search_query)
** User clicks Woogle search result (woogle_result_click)
** Icons displayed to the user on Woogle search results (woogle_search_icons)
** Icons displayed to the user on red links (woogle_redlink_icons)

A logging target should at least log its code (see values above) to allow analysis. Additional parameters may be concatenated to the log string, separated with ";".

The actual logging is implemented in [https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/addons/Instrumentation/WoogleInstrumentationLogger.php WoogleInstrumentationLogger]. In this implementation, the log is text-based ([http://en.wikipedia.org/wiki/Comma-separated_values CSV-formatted]). Each line contains a timestamp, the pseudonymized user id (see above) and an arbitrary string defined by each logging targets itself.

== Data analysis ==
The instrumentation framework does currently not include any automated data analysis. The usual worflow is to import the [http://en.wikipedia.org/wiki/Comma-separated_values CSV-formatted] log files to statistical analysis tools such as [http://en.wikipedia.org/wiki/R_%28programming_language%29 R], [http://en.wikipedia.org/wiki/Spss SPSS] or any spreadsheet calculator.

== Installation ==
The code for the instrumentation addon can be obtained from SVN (https://waves1.fzi.de/svn/waves/trunk/Woogle4MediaWiki/addons/Instrumentation/).

Instrumentation mode can be enabled by setting <code>$config['instrumentation']</code> in <code>extensions\Woogle\addons\Instrumentation\Instrumentation.php</code> to "true".

Per default, only user logging in with an own account are targeted by the instrumentation. Setting <code>instrumentationAnon</code> true will also log anonymous users.

TeamWeaverWiki - New pages [en]

Appstoremining

Appstoremining/

BR/interviews

Next!

Woogle4MediaWiki/Instrumentation addon