IIT Bombay Techfest - Tata Communications Datathon | DPhi

Problem Discussion
#1
This is a companion discussion topic for the original entry at https://dphi.tech/challenges/165
#2

I am unable to submit my predictions.
When I am submitting it is showing cannot rehsape array of size.
Please help and guide me on this issue.

#3

@dt-214238 could you please try making the submission now? it should start working

#4

@chanukya The leaderboard is based on mape right?

#5

I think there are some issues in evaluating the scores.

#6

@tac_o_3152 a custom evaluation metric has been used for this challenge. The custom metric uses MAPE, we’ll share that code soon.

#7

@tac_o_3152 you can the code here. or here

#8

@chanukya
I think there is a mistake in your mape calculation, the predicted value can be 0 , and according to the notebook provided, the error is just ignored when predicted value is 0.

#calculating mape score
overall_mape=0
overall_mape_sum=0
for i in range(0, validation_df.shape[0]): #looping through the regions
  
  region_mape=0

  for j in range(1,8): #looping through the months
    actual_value=validation_df.iloc[i][j]
    predicted_value=predictions_df.iloc[i][j]
    if(actual_value!=0 and predicted_value!=0):
      mape=((actual_value-predicted_value)/actual_value)*100
      if(mape<0):
        mape=mape*(-1)
      region_mape=region_mape+mape
    else:
      pass

  mape_df.at[i,'Mape_score']=region_mape
  overall_mape_sum=overall_mape_sum+region_mape

#print(overall_mape)
#print(overall_mape/validation_df.shape[0])
overall_mape=(overall_mape_sum/validation_df.shape[0])
#9

@chanukya Just a suggestion can you please check the result against the mean_absolute_percentage_error library function in sklearn.metrics module as a reference?

#10

@tac_o_3152 yes, you are right with the observations. However, the metric calculation seems fine. This is something that was customized by the SMEs from the main partners of the Datathon. In general, MAPE comes with certain constraints and in order to overcome them certain conditions were written.

#11

@dt-211418 please don’t use the mape function in sklearn, it won’t provide you with the correct result. Instead it is better to run the predictions on the customized eval function that we shared on the challenge page.

#12

qwe
qwe899×219 9.34 KB

I am getting this error. Can anyone help?

@chanukya

#13

@chanukya
The only constrain for the basic implementation of mape is that actual value must not be zero.
However in this implementation, the prediction is ignored when the actual value is zero and also in cases when the predicted values is zero.

Hence a submission with all zero entries can get a mape score of 0 (topping the leaderboard).

#14

Yes, you are right, we are aware of it. All the top notebooks will be verified before we announce the final result. In addition, we expect participants to be responsible and submit valid submissions as they will be vetted later.

#15

@dt-216177 seems part of the entries in your submissions is empty which is causing this error. could you please check that?

#18

@chanukya Is there any way to change the team name? My name needed some corrections.

#19

@dt-216177 please write to support@dphi.tech

#20

@chanukya sir, Seems there is a glitch in the evaluation, null scores are rank 1, pls look into it

#21

@david123 thanks for bringing it up again, as mentioned in the previous discussions, the evaluation metric is tuned in such a way that 0 are ignored as per the custom eval metric defined by the organizers. Hence, you are seeing this issue. We recommend you to mark your final submission with correct prediction files. Also, please note: All the top notebook submissions will be validated by an expert so any false positives will be removed from the final ranking. Hope this helps.

#22

@chanukya sir, the submission deadline has been updated on the official website of IIB here: TATA Comm Datathon | Techfest, IIT Bombay
When will it be updated on this portal also?