Update V1.8 was there a beta test?

Discuss The Products
,
21

I agree completely with @tin

Actually, I am surprised that we need to give advises to a phone maker about how to beta test their software and how to handle their software contractors. But ok, let’s say it again :

  • You decide what goes in your OS. Not your contractor. Your contractor might make constructive proposals but it should be the exceptions. It implies of course that you know what should be added, modified or removed from your software.
  • Your contractor must QA its software extensively.
  • You should beta test the software with a subset of users, following the exact procedure that will be used and using the exact binary that will be made available. Iterate with the contractor to fix the bugs and beta test again each version.
  • Don’t release until you are satisfied. Of course, the bigger the update the longer you will have to wait. This is why you should deliver often small updates. Not like you are doing now.

As for the “users should have done this or that, read the manual and been smart” :
You might not like it but your users are always correct. I am a software engineer and I know how frustrating it is when a user does stupid things with your software. But why did you let him/her do it ? make you software fool proof. Do not consider anything obvious to him/her.

I hope that the software side of the FP will mature in the coming months and years !
Barth

22

I understand all replies here, and like to address this: Fairphone is hiring Android-developers. And yes, maybe some of you think this is too late. But if you are or know someone capable of joining the Fairphone-team, do not hesitate! :smile: Btw, I don’t work for Fairphone, if I had my bachelor in computer science I wouldn’t hesitate for a second, however I’m still studying :smiley:

23

@tin and @Barth I agree with making the software fool proof and we will work hard on maturing the software.

I read through both of you posts and your points are clear on the more negative aspects and strategic level. Seeing both of you have experience in software/beta testing, would you be willing to add what kind of methods/systems/setup you would see in a “perfect BETA test” for a next update? How do you see the practical setup of an extended beta program for Fairphone owners - to help us test new releases?

24

@Marco I appreciate your comment and the positive way you receive our criticism.

I think that you answer to a certain extent your own question. A good beta test involves end users, if possible well motivated people but that is not a problem in our community. First step would be to open a page where people can register. Once you have a beta ready, contact them with a link install it. I don’t know well the android platform from a developer point of view and thus can’t advise on how to provide this installer only to selected people.

The beta testers should have a way to report via a special way to the developers, and maybe a dedicated (open? not sure) forum.

I hope it can help you.

25

These are some great ideas and I’m sure the Fairphone will receive them with open arms :slight_smile: Thank you @Barth !

26

I’d be up for beta testing the next upgrade. I’d have done so already now, if I had known …
Having a “Beta” topic would be nice, and I’d prefer that to having to check yet another information source, yet another forum to register to.

One idea could be to have an option for the updater app to receive beta updates or not, and have it disabled by default. This way the entire roll-out procedure could be checked (including how it is handled by the update app), yet only users who knowingly changed the option would receive the upgrade, while non-testers do not get bothered.

27

Fast thoughts:

  1. “Perfect” doesn’t exist.
  2. Build a beta tester group and give them the opportunity to install
    all updated OS versions (nightly?) via a “normal” update process -
    same as the release version will have.
  3. Build a bug report platform (Bugzilla eg) where beta testers can
    describe as detailed as possible what they have done and what they got.
  4. Take all these reports in consideration - do not say “its only
    cosmetic” and try to verify the reported bug or problem with your
    phones and development systems.
  5. Try to fix the bug --> new build
  6. On a mailing list all fixed bugs/problems are reported
  7. At the end of this process --> release version

(Sorry, English is not my mothertongue, hope it’s clear enough)

28

The main homescreen, which comes up, when you push the home button, is always the one in the middle. You can drag your preferred homescreen to the middle, so to have the other screens on both sides of the main screen. Edit: This is being discussed in this topic.

Actually you can have “secret rooms” on the Discourse platform, e.g. a room only visible for the people, which applied for the beta test group.

30

I like your idea of having an option in the updater. Also, having a Beta topic in the forum is good for me.

31

Hi @ all, I too appreciate the idea of beta testing and would join a team of beta testers.

I wonder if anyone has noticed the following:

When you had the version 1.6 on your FF, the automatic update updated to VERSION 1.7 !!!

If you wanted version 1.8, you had to START THIS UPDATE AFTER THE AUTOMATICAL UPDATE TO 1.7 YOURSELF.

I did so, and now I have 1.8. Some things I did not like about the 1.7 Update (which I thought would be the 1.8 Update) are gone now that I have the (real) 1.8 Update.

I will check if I find this observation elsewere in the forum, but I have not found it so far.

32

Hi all,

Please, let me apologize first: English is not my natural language. For this reason my explanation may sound rude, but it’s not my intention in any way! :smile: Also, it expresses my humble opinion.

I agree with @tin and @Barth, quality assurance is something to take in serious consideration.

Right now I’m working as software quality assurance analyst and developer (when needed), and testing is a crucial part of software development. Having a beta test phase is a very good initiative, but testing process has to begin a lot earlier.

Let’s imagine those scenarios when a beta version is released:

  • The update bicks the Fairphone. And the Fairphone is the only phone the beta tester has.
  • Counting all the beta testers, no one has one of the FP models and/or a FairphoneOS version. Those scenarios won’t be tested.

Also, the testing process would be very random. If there is no strict test plan, there’s no assurance that the testing was deep enough, only the superficial bugs would appear.

Of course, having a quality assurance department with ten people can’t guarantee in any way that the software will be bug free. Also, there is no way to test, for example, bluetooth compatibility with 100% of devices, that’s just impossible. But quality assurance is needed to minimize issues as much as possible, and it needs a great effort and a lot of time.

I checked the team profiles and there’s no software quality assurance role. Fairphone is hiring Android developers, but it’s not the profile they should be looking for (for this matter), as KwameCorp is developing the software (please correct me if I’m wrong). A quality asurance profile is very different from a developer profile.

I don’t know if KwameCorp has testers and QAs in their development process, but even if they do, quality has to be checked deeply from Fairphone.

I hope I explained myself well, and sorry for the novel I just wrote here :stuck_out_tongue_winking_eye:

And let me congratulate Fairphone team, they are doing a great job. Hardware and software development is very, very difficult, and there is quite a long journey ahead.

Cheers!