Week #5 #
Feedbacks: #
Feedback collection plan #
After collecting feedback last week, we identified several ways to make the process more convenient for users and improve our analysis of their responses. As a result, we have used a slightly different strategy for gathering feedback from clients this week.
Helpful tools we used:
- Online Forms: We utilized Google Forms to create and distribute surveys.
- Data Analysis Tools: We used Google Sheets to organize and analyze the feedback data.
Survey Questions for Telegram bot:
Usability:
- How easy is it to navigate through the bot’s interface? (Scale: 1-5)
- How intuitive do you find the process of reporting a problem? (Scale: 1-5)
- Did you encounter any difficulties while using the bot? If yes, please describe.
- How long did it take you to complete a report using the bot? (Multiple Choice: <1 min, 1-3 min, 3-5 min, >5 min)
- What features or steps did you find most useful or confusing?
Functionality:
- Have you successfully reported a problem using the bot? (Yes/No)
- Did you receive notifications about problems in your building? (Yes/No)
- How timely were the notifications you received? (Scale: 1-5)
- Were the problem reports you received clear and understandable? (Scale: 1-5)
- What additional features would you like to see in the bot?
Overall Satisfaction:
- How satisfied are you with the bot’s performance? (Scale: 1-5)
- Do you feel that the bot has sped up the process of problem reporting and resolution? (Scale: 1-10)
- Would you recommend this bot to other dorm residents? (Yes/No)
- Please provide any additional comments or suggestions for improvement.
Survey Questions for website:
Usability:
- How easy is it to navigate the website’s interface? (Scale: 1-5)
- How intuitive do you find the process of searching for specific parameters? (Scale: 1-5)
- Did you encounter any difficulties using the website? If yes, please describe.
- How long did it take to find the information you were looking for? (Multiple Choice: <1 min, 1-3 min, 3-5 min, >5 min)
- What features or sections did you find most useful or confusing?
Functionality:
- Have you successfully used the graphs and statistics on the website? (Yes/No)
- Are the search parameters comprehensive and useful? (Scale: 1-5)
- How accurate and helpful are the statistics provided? (Scale: 1-5)
- What additional features would you like to see on the website?
Overall Satisfaction:
- How satisfied are you with the website’s performance? (Scale: 1-5)
- Do you feel the website has improved your ability to manage dormitory issues? (Scale: 1-10)
- Would you recommend this website to other dormitory staff? (Yes/No)
- Additional comments or suggestions for improvement.
Conducted user surveys or feedback sessions #
Results of bot feedback session
Our target audience consisted of 31 students, whom we asked to provide feedback on our product. They were instructed to send 1-2 reports to our bot. Following this, they rated the ease of sending reports, receiving notifications, and their overall satisfaction with using the bot. We then analyzed their responses from the Google Forms below.
Usability:
If you encounter any difficulties while using the bot, please describe.
- Old reporting windows are still active. But this is minor issue, does not break anything except command that can be sent at the state of optional comment
- The time to receive a confirmation code on email is longer than in other services. Sometimes, buttons pressures are processed for a long time
- Slow email confirmation, sometimes slow button reaction
- It’s not a difficulty, but I want to see a list of subscriptions somewhere. Just don’t want to go through all the possible report options
- I’d like to have a clearly understandable navigation after I log in
- It could be great to have a bot feedback when I leave comment on problem
- “Alerts enabled” — confusing little bit. May be name it as disable alerts?
- When I press /start again after using the bot nothing happens
- It could be better to have buttons instead of handling everything with / commands
- Meniu buttons are essential for good UX in Telegram bots. I wish you will add them as soon as it is possible
- Find subscriptions for the specific dorm
- Some my messages disappear after I send them. It is not clear to me should are they being deleted
What features or sections did you find most useful or confusing?
- Optional report commenta may help to understand the core of problem imho
- Useful that I can subscribe to obtain notifications about specific building
- It is useful that I can just choose the labels and floors (don’t have specific format for writing)
- Notification system is very helpful
- Reports summarization
- Choice between descriptive and usual report
- I find the bot very helpful! AI integration is awesome, the project is a success!
- Little bit inconvenient to unsubscribe from alrrts
- Get notifications about problems
- Navigation to reporting the issue
- Selecting the dorm is implemented very good! :)
- Buttons (user experience in general)
- It is confusing that some commands (like /start) that are send to the bot are being deleted. I want to have full history of my dialogue with the bot.
- Inline buttons for choosing the dorm and kind of trouble are implemented very well.
- Choosing the dorm using inline buttons is very cool feature!
- Automated reports.
- Do not delete messages
- removing users’ messages?why?
Functionality:
What additional features would you like to see in the bot?
- alerts were received multiple times but this is probably ok
- I don’t know why my reports were not sent to me, but may be it is not enough
- Maybe it will be good to upload the photo about problem
- Tracking of my report (is it approved or no)
- It could be cool to maybe have a map of the dorms and report locations on it
- May be map of failures
- Maybe notifications not only about problems that have already happened but also notifications about upcoming changes (about water shutdown and etc)
- Buttons for more intuitive navigation
- I received alert multiple times
- Buttons as I said
- Option to attach photos to reports nothing, i like it
- Messages in reports could be more concise and informative
- Do not delete messages update info on reports (if there are new ones)
- Results on resolving reported issue
Overall Satisfaction:
Please provide any additional comments or suggestions for improvement.
- now bot alerts are not used anyhow, but they could be e.g. sent through dormitory representative to adminitstration
- May be button interface in keyboard?
- I like this bot! I think you should develop it for future.
- Very good idea, because in usual way I know about outrages very late
- Do such system not only for dorms but for university or maybe for city
- Good project
- Can this bot really report stuff to dorm admins?
- I would like to have a feedback from bot after leaving the message on problem in dorm.
- Sometimes requests are processed slowly
- Fix the case with sending /start
- The idea is very interesting, I wish it will speed up the processing of fixing troubles in dormitories! Good luck with the implementation:)
- Simplify the subscription system
- this approach can lead dorm issues reporting to absolutely new level! happy to see it
- I have used your bot about a week ago and I see that user interface has been improved very much. Good job, fellows!
- Messages should remain
- I think it was better to make a website instead of Telegram bot
- don’t remove messages, it is strange behavior
Results of website feedback session
Since our website will be used by the university administration, we could only gather two reviews. However, even this small amount of feedback has significantly contributed to the development of our product.
Usability:
Functionality:
Overall Satisfaction:
Analyzing feedback, identifying and prioritizing issues #
By conducting user surveys and feedback sessions, we obtained useful information about usability, functionality, and overall satisfaction with the Telegram bot reporting problems in the dormitory. Below are the key findings and parts for improvement based on user feedback.
Bot Survey Analysis
Usability:
Ease of Navigation:
- Average Rating: 4.26/5
- Comments: Most users found the bot easy to navigate, though a few students had difficulties.
Intuitiveness of Reporting Process:
- Average Rating: 4.26/5
- Comments: Users appreciated the straightforward process but suggested including more examples of common issues.
Encountered Difficulties:
- 45.2% of users reported difficulties. Users reported several issues with the bot, including old report windows remaining active, delays in receiving email confirmation codes, and slow responses to buttons. They also noted the need for a subscription list, clearer navigation after logging in, and feedback to the bot when leaving comments on issues. Some found the “Alerts Enabled” label confusing and suggested renaming it. Other issues included lack of response to pressing “/start” again, favoring buttons over /commands, the need for menu buttons to improve UX, difficulty finding subscriptions for specific dorms, and messages disappearing.
Time to Complete a Report:
- <1 min: 48.4%
- 1-3 min: 32.3%
- 3-5 min: 19.4%
- 5 min: 0%
- Comments: Most users could complete a report quickly.
Useful/Confusing Features:
Useful Features:
- Optional report comments for better problem understanding.
- Subscription to notifications for specific buildings.
- Label and floor selection without specific format requirements.
- Helpful notification system.
- Summarization of reports.
- Choice between descriptive and usual reports.
- AI integration in the bot.
- Notifications about problems.
- Navigation to issue reporting.
- Good implementation of dorm selection.
- General user experience with buttons.
- Inline buttons for dorm and trouble selection.
- Automated reports.
Confusing Features:
- Difficulty unsubscribing from alerts.
- Commands like /start being deleted, preventing full dialogue history.
- Messages being removed after sending, causing confusion.
Functionality:
Successful Problem Reporting:
- 100% of users successfully reported problems.
- Nobody faced issues with report submission due to technical glitches or misunderstanding commands.
Receiving Notifications:
- 80.6% received notifications about problems.
- 19.4% did not receive notifications, likely due to incorrect subscription settings.
Timeliness of Notifications:
- Average Rating: 4.1/5
- Comments: Notifications were generally timely, though a few users experienced delays.
Clarity of Problem Reports:
- Average Rating: 4.3/5
- Comments: Reports were clear, but some users requested more detailed information about the reported issues.
Additional Features Suggested:
- Photo Upload: Option to attach photos to reports.
- Report Tracking: Ability to track the status of reports (approval or resolution).
- Map Integration: Map of dorms and report locations, as well as a map of failures.
- Notification Enhancements: Notifications about upcoming changes (e.g., water shutdowns), not just existing problems.
- Improved Navigation: More intuitive navigation buttons.
- Alert Frequency Control: Addressing the issue of receiving multiple alerts.
- Message Management: Retaining messages instead of deleting them.
- Report Summarization: More concise and informative messages in reports.
- Update Information: Providing updates on reports if there are new developments.
- Resolution Feedback: Results on resolving reported issues.
Overall Satisfaction:
Satisfaction with Bot’s Performance:
- Average Rating: 4.6/5
- Comments: Users were generally satisfied, noting improved efficiency in problem reporting.
Perceived Speed of Problem Resolution:
- Average Rating: 7.8/10
- Comments: Users felt the bot sped up the reporting process, though some noted that resolution speed depended more on maintenance staff efficiency.
Recommendation to Others:
- Yes: 96.8%
- No: 3.2%
- Comments: Most users would recommend the bot, but those who wouldn’t cited occasional technical issues as their reason.
Additional Comments/Suggestions:
Bot alerts could be utilized more effectively by sending them through the administration dorm representatives. A push-button interface in the keyboard would improve usability. The bot is appreciated, and there are suggestions for further development, possibly expanding the system to the entire university or city. It is recommended to establish feedback to the bot after reporting a problem, speed up the processing of requests and fix the problem with sending “/start”. It is also suggested to simplify the subscription system and make it so that messages are left rather than deleted. Overall, the project is rated as promising and shows significant improvements to the user interface. Users are optimistic that it will speed up the resolution of dorm problems and consider it a good project that can take the dorm problem notification system to the next level.
Future work:
High priority
- Improve Email Confirmation and Button Response Times: Address delays in email confirmations and slow button responses to enhance user experience.
- Intuitive Navigation: Add more intuitive navigation buttons and a push-button interface in the keyboard.
- Concise and Informative Reports: Ensure that messages in reports are concise and informative.
- Notification Enhancements: Include notifications about upcoming changes (e.g., water shutdowns) and provide timely updates on reports.
Middle priority
- Map Integration: Integrate a map of dorms and report locations, as well as a map of failures.
- Subscription List and Simplification: Implement a subscription list and simplify the subscription system to make it more user-friendly.
- Feedback Mechanism: Establish a feedback mechanism to the bot after users leave comments on problems.
- Implement Photo Uploads for Reports: Allow users to attach photos to reports for better problem identification.
- Track Report Status: Provide functionality for users to track the approval and resolution status of their reports.
Low priority
- Retain Message History: Make sure messages are not deleted and retain the full dialogue history with the bot. Resolve /start Command Issue: Fix the issue where pressing “/start” again does not yield any response.
- By implementing these changes based on user feedback, the bot can significantly improve its usability, functionality, and overall user satisfaction.
By addressing the feedback and implementing the suggested improvements, we aim to enhance the overall user experience and effectiveness of the dorm trouble reporting Telegram bot.
Website Survey Analysis
Usability:
Ease of Navigation:
- Average Rating: 4.0/5
- Users generally found the website easy to navigate, highlighting intuitive navigation through the use of icons and accent colors.
Intuitiveness of Searching for Specific Parameters:
- Average Rating: 4.0/5
- Mixed feedback was received regarding the intuitiveness of searching for specific parameters. One user found it highly intuitive, while another user rated it moderately.
Encountered Difficulties:
- No difficulties were reported by both users in the provided feedback.
Time to Find Information:
- <1 min: 1 user
- 1-3 min: 1 user
- Most users could find the information they were looking for quickly.
Useful or Confusing Features:
Useful Features:
- The reports section is crucial and well-navigated through icons and accent colors.
- Graphs and statistics were found useful by the users who successfully used them.
Confusing Features:
- The administrators’ section was deemed unnecessary by one user, as they questioned the need for all administrators to have access to information about other admins.
Functionality:
Successful Use of Graphs and Statistics:
- Both users successfully used the graphs and statistics on the website.
Comprehensiveness and Usefulness of Search Parameters:
- Average Rating: 3.5/5
- Feedback indicates that search parameters are generally comprehensive and useful, though there may be room for improvement.
Accuracy and Helpfulness of Statistics:
- Average Rating: 4.5/5
- Users found the statistics provided to be accurate and helpful.
Additional Features Suggested:
- Sorting by floor or category: One user suggested implementing sorting features to enhance usability.
- Live graphics and immediate repairer call: Another user recommended the addition of - live graphics on the website and the ability to call a repairer directly from the report information.
- Confirmation and Notification System: Adding a feature to confirm problems directly from the report page and notify the sender about the problem resolution.
Overall Satisfaction:
Website’s Performance:
- Average Rating: 4.5/5
- Users were generally satisfied with the website’s performance.
Improvement in Managing Dormitory Issues:
- Average Rating: 9.0/10
- Users felt that the website has significantly improved their ability to manage dormitory issues.
Recommendation to Others:
- Both users would recommend the website to other dormitory staff.
Additional Comments/Suggestions:
One user appreciated the user interface, describing it as good and understandable. Another user requested the inclusion of live graphics and the ability to immediately call a repairer from the report information.
Future work:
High priority:
- Implement Sorting Features: Allow sorting by floor or category to improve the usability of the website.
- Add Live Graphics: Integrate live graphics on the website to enhance the user experience.
- Repairer Call Feature: Enable immediate calling of a repairer from the report information to streamline the problem resolution process.
Low priority:
- Confirmation and Notification System: Develop a feature to confirm problems directly from the report page and notify the sender about the problem resolution.
- Review Administrator Section: Evaluate the necessity of the administrators’ section and consider restricting access to information based on user roles to address concerns about its usefulness.
By addressing these key areas based on user feedback, the website can further improve its usability, functionality, and overall user satisfaction.
Roadmap: #
We will plan to implement the following enhancements:
- Extra Statistics with ClickHouse: This will allow for more detailed and comprehensive data analysis.
- Horizontal Scaling with ClickHouse: To improve the system’s scalability and performance.
- ML-Based Failure Detector: Implement machine learning algorithms to detect failures proactively.
- Reward System: Introduce a reward system to motivate users to submit accurate and helpful reports.
Weekly Progress Report: #
This week has been crucial for our project as it marks the final stage of development. After working on the creation of the product, we are now focusing on polishing it to perfection. Speaking about the changes:
- Bot Enhancement: After taking the first, ‘hot’ user surveys, we have conducted a major refactoring of the user interface. We changed the style of our buttons, creating a more visually appealing and, more importantly, a clearer and more concise interface. Based on the user’s flow, we are trying to make the use of the application as smooth as possible. Check it by yourself, there is the link: BOT
- Fault Alerting Mechanism: At the stage of creating a smart algorithm, we have finally reached the ‘golden ratio’. The service can now use data from users to analyze it using a self-developed mechanism and investigate the possible reasons. It can then summarize the overall feedback and notify subscribed users. Simply put, the system can now function properly as an isolated one, means without any human intervention. The use of AI has replaced the need for manual labor.
- Frontend Website: The well-designed website is approaching the production stage. We have tested a large number of features, such as graphics and an admin panel controller, and we are planning to analyze the feedback we have collected in order to make any necessary changes and integrate them properly into our ecosystem. This will help us bring our project closer to completion.
- Backend: The essential part of the entire system is now brand new. After all the testing activities were completed, we can now introduce the fully functional API application with a coverage of nearly 90%. Additionally, the backend has now implemented SonarQube code linter, which analyzes code on each pull request submitted to the dev branch. Due to our careful approach to writing code, there are no issues with this at all.
- Database: It is hard not to mention another important point: now, DDoS attacks are nearly impossible. Our ecosystem will not process reports that are sent twice in a row with identifying meta information within N minutes (a configurable parameter). This is all implemented without any changes to the source code, simply by using the capabilities of the PostgreSQL database.
Challenges & Solutions #
The challenges of this week are connected with the previous ones because the reason for some is not clear enough. Saying more concretely:
- Database Server Time Zones: There are problems with the timezone of the PostgreSQL database’s server. It seems that the server has an incorrect timezone, but we need to know more about it to answer the raised questions.
- Frontend & API Connection: There are some issues with the IU virtual machines and the web application due to the network configuration at our university. For some reason, connections cannot be established to certain endpoints while they are working perfectly for others.
Conclusions & Next Steps #
We need to make the system as stable as possible and provide the functionality to our direct users for further investigation and enhancement iterations after the production release.
- Complete the website development process and integrate it into the ecosystem.
- Add dashboards to Grafana to make it easier for admins to use this analytics tool.
- If possible, create a CI/CD pipeline that will automatically integrate new features into our system without manual intervention from developers.
- Write up the necessary documentation for our systems that can provide crucial and, at the same time, brief information about technologies.