Colnect, Connecting Collectors. Colnect offers revolutionizing services to Collectors the world over. Colnect is available in 63 languages and offers extensive collectible catalogs and the easiest personal collection management and Auto-Matching for deals. Join us today :)
Sunday, April 12, 2009
When Web Crawlers Attack
On my site for collectors, I have created a pretty extensive robots.txt file to prevent some nicer bots from scanning parts of the site they shouldn't and blocking semi-nice bots. In addition, server rules to block some less than nice bots out there were added.
The biggest problem left unanswered is what to do when the supposedly nice bots attack your site. The web's most-popular bots is probably GoogleBot, create and operated by Google. Obviously, it brings traffic and is a good bot that should be allowed to scan the site. However, more and more frequently I see that the bot is looking for more and more URLs that NEVER existed on the site. Atop of that, since the site supports 35 languages, the bot even made up language-specific URLs. For some reason, it decided I should have a /en/phone page and so it also tries to fetch /es/phone, de/phone and so on.
So why is that so annoying? Two main reasons:
1/ It appears in my logs. I check these for errors and end up spending time on it.
2/ The bot is not giving up on these URLs although a proper 404 code is returned. It tries them over and over and over and over again.
Any suggestions? Seems to me that modifying robots.txt with 35 new URLs each time GoogleBot makes up a URL isn't the easiest solution.
The problem is not unique to GoogleBot. I have completely blocked Alexa's ia_archiver which is making up URLs like crazy.
Are there any reasons for inventing NEVER-existing URLs? Probably broken HTML files or invalid links from somewhere. Sometimes, wrong interpretation of JavaScript code (do they really HAVE TO follow every nofollow link as well???) seems to be the reason.
2009/04/15 - Read the update
Tuesday, April 7, 2009
Colnect Rising on Compete
Though I update about trends in site metrics for Colnect, I'm not really sure what they mean as they don't always coincide with my Analytics results. You're welcomed to check Colnect's rankings on Compete. It has risen 34% in the last month. Pretty nice :)
Sunday, April 5, 2009
GMail turn 5 - still BETA??? Colnect will not follow.
Google even created the 'beta' mark trend in logos of companies and services.
I personally find it rediculous and unfair to the customers. Of course products sometimes fail but we cannot abuse the term "BETA" for 5 (FIVE!!!) years.
Colnect has been marked as beta for less than 6 months since it went public before all key features were ready and prior to proper testing. Raising a site from grass-roots up is not a simple task. However, as of today, since Colnect is relatively stable and many of its key features (a lot more is to come but I'll elaborate on that another time) are ready and publicly available, the BETA mark will be removed.
Yes, my system may sometimes fail. Yes, it's not as perfect as I'd like it to be. However, it's public, it's working, it makes many people using it happy so it's not a beta anymore.
Thursday, April 2, 2009
Buying and selling collectibles
Buy List / Sell List
These lists are available with Premium Membership. Unlike Custom Personal Lists, collectibles added to these lists appear on the Collectors inventory information section of each single collectible item page.
When adding collectibles to these lists it is best to put the relevant price in the public note box. We suggest using world-popular currencies and use their 3 letter code rather than symbol. Example: USD is always US dollar, but the $ sign has different meaning in different countries.
NOTE! Prices you quote must be valid. You may add details regarding trades on your personal page under My Account. Complaints received regarding invalid prices (for example: you offered to sell an item for a certain price but later asked for a higher price) will be investigated. If you are found dishonest, your Colnect account may be deactivated without any refunds.
Japanese and Lithuanian languages added
Translations on Colnect are performed manually by volunteering translators who are members of the site. Whenever a phrase is not properly translated they can translate it easily. It's all explained here.
A recent addition is the use of automated suggestions. When a phrase has not yet been translated, it'll first be translated with an automated suggestion. An icon telling the translator he should translate (or confirm) that phrase still exists. The use of suggestions is intended for the period of time after a new content is published on Colnect (which is quite often) until a translator actually gets to translate it.
Yes, automated translations sometimes suck really bad. For example "FREE trial - 1 month" had a Hebrew suggestion that can be translated back to English as "Free trial - 1 year". What?!?! How did a month become a year? That is quite dangerous and I hope these mistakes are not too frequent. I hope that the automated suggestions many times "get over the net", meaning they are understood by the reader although acknowledged as improper language use.
Japanese is currently the only language for which Colnect yet has no translator and so we rely on the automatic suggestions. It's a sort of pilot to see if it can attract Japanese collectors and hopefully one of them will agree to become a translator. If this experiment succeeds, other languages may be added this way. A warning message will be displayed with languages that are not completely manually translated.
You're welcomed to check Transposh for translation solutions.
Tuesday, March 31, 2009
Twitter fails as a promotional tool?
* Colnect news Twitter @colnect
* A personal Twitter for Colnect's manager @AmirWald
* Automated feed reporting new collectibles in Colnect's catalogs @ColnectCatalogs
* Automated feed on Colnect's catalog edits @ColnectEdits
During these days 28 visits came to Colnect from Twitter, a meager amount in comparison to the number of "followers" and energy invested. The bounce rate (visitors seeing a single page and leaving the site) was incredibly high as well. In comparison, a few posts on a relevant forum resulted in hundreds of relevant visits (with much lesser bounce rate).
It seems a lot of people use Twitter to self promote and so it's more of a bubble where "followers" is a rough indication of the number of people who will actually read anything of what you write. My guess is that for most people, a small percent of their "followers" actually read more than 5% of their tweets. Though some people think of it as a useful personal tool, it doesn't seem like they dominate Twitter.
Though less than a week may be a too short amount of time for a verdict, results so far are very unsatisfying. In the future, the automated feeds may be of use to some of the addict collectors on Colnect. Let's see what the future brings.
Wednesday, March 25, 2009
Colnect on Twitter
Easier than blog posts? Obviously...
Useful? hmmm.....
Anyway, you're welcomed to follow the official Colnect on Twitter
All public updates regarding Colnect may be there before anywhere else.
Tuesday, March 24, 2009
Custom Personal Lists on Colnect
Custom Personal Lists
In addition to the common inventory lists on Colnect (Collection / Swap / Wish), premium members can create more lists. When custom lists are added, you can add and remove item to and from them in the same way you do for the common lists.
Common Uses
- Reserving items for a specific swap
- Offering many item for a single sale
- Marking items you wish to attend to later
- Creating a list to be shown to collectors on other sites
Monday, March 23, 2009
Gmail: back to the future
Here's the attached picture. My interface is in Spanish so "minutos antes" means "minutes ago".
Friday, March 20, 2009
Biggest phone cards catalog for collectors - 142,000+ phone cards listed
Coins catalog: over 14,000 coins
Colnect's coins catalog is currently the world's biggest freely available resource for coin pictures and information.
Wednesday, March 18, 2009
Colnect rises on Compete / Quantcast / Alexa
I've been updating about Colnect's rank rising on Alexa and have now decided to include links to other rankings. So you can check out Colnect's rankings on Compete and Quantcast and start wondering what all these ratings are actually worth. So although it's nice to see Colnect gains momentum on all non-related meters, it's interesting to note the difference between what they report and between the reality. By "reality" I usually mean my reports from Google Analytics and, when really bored, the server's logs can be inspected. There's little connection between the real graphs and the estimated ones by these services. Colnect has been growing and growing throughout. Sometimes slower, lately faster. The graphs by Compete and Quantcast actually show a very inaccurate picture. Oh well...
Monday, March 16, 2009
Collectors premium services on Colnect
The announcement has been made:
"Premium membership gives you access to several features that will enhance your Colnect experience. If you regularly use Colnect, please consider supporting us by becoming a premium member."
Currently, not many premium services are offered but these will be added with time according to collector demands.
Happy collecting :)
Sunday, March 8, 2009
Colnect's Alexa Rating Keeps Climbing
The ranking is coherent with Colnect's internal report, showing a significant increase in traffic during the last months.
Wednesday, March 4, 2009
Email Anti-Spam Tip
An Example
Today I've received this unsolicited (SPAM) email trying to frighten me into buying colnect.tw, colnect.cn and so on. Here it is:
From: kevin.wu
(If you are NOT CEO,please forward this to your CEO, because this is urgent.Thanks.)
Dear CEO,
We are the department of registration service in China. we have something need to confirm with you. We formally received an application on March 3, 2009, One Japan company called "Path soft investment Corp" are applying to register (colnect) as internet brand name and domain names as below (colnect.net.cn colnect.org.cn colnect.mobi colnect.asia colnect.hk colnect.cc colnect.tw etc.).
After our initial checking, we found the internet brand name and these domain names being applied are as same as your company’s, so we need to get the confirmation from your company. If the aforesaid company is your business partner or your subsidiary company, please DO NOT reply us, we will approve the application automatically. If you have no any relationship with this company, please contact us within 15 workdays. If out of the deadline, we will approve the application submitted by "Path soft Corp" unconditionally.
We would like to get the affirmation of your company,please contact us by telephone or email as soon as possible.
Best Regards,
Kevin Wu
Senior Director
TEL: +86 21 69929440
Fax: +86 21 69929447
Website:www.qpnic.org.cn
Shanghai QPNIC Web Property Solutions Limited
Now, this email was sent to management@ and webmaster@ (both of them at my domain colnect.com). However, since I never give these email addresses to anyone, it means that every email I get there is SPAM. Other common names such as info@ and contact@ and others are frequently used.
A short search on the Internet affirmed my suspicion of a scam, when I found this blog post and that one.
What About My Private Email?
Well, you can easily get yourself a free domain on any service that would freely forward your email (such as cjb.net). Then, whenever you register a website, make up an email site_x@mydomain.cjb.net and use it to register the site. You would always be able to receive such emails but when this address becomes 'dirty' (starts receiving much spam), you can filter out all emails coming to it. It's a better solution than one-time emails since sometimes you do actually want to allow the site to later contact you. It is also a completely legitimate email address.
Symfony: Error Logging Hack
The following hack can be has been customized for my needs but you can change it to your preferences. It'll change the output Symfony places in the PHP error log file.
What the Hack Does?
A boring Exception such as:
[04-Mar-2009 17:20:25] Action "coins/collect" does not exist.
Will become:
[04-Mar-2009 17:20:25] CODE[0] MESSAGE[Action "coins/collect" does not exist.]
FILE[.\config_core_compile.yml.php] Line[715]
REQUEST[/it/coins/sdlk] REFERER[]
AGENT[Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6] ADDR[127.0.0.1]
How To?
Find sfException class (should be /symfony/lib/exception/sfException.class.php) and add the following method:
public function getMessageFull() {
$exception = is_null($this->wrappedException) ? $this : $this->wrappedException;
try {
$sReq = isset($_SERVER['REQUEST_URI']) ? $_SERVER['REQUEST_URI'] : '';
$sRef = isset($_SERVER['HTTP_REFERER']) ? $_SERVER['HTTP_REFERER'] : '';
$sUserAgent = isset($_SERVER['HTTP_USER_AGENT']) ? $_SERVER['HTTP_USER_AGENT'] : '';
$sRemoteAddr = isset($_SERVER['REMOTE_ADDR']) ? $_SERVER['REMOTE_ADDR'] : '';
$sErrMessage = "CODE[".$exception->getCode().'] MESSAGE['.$exception->getMessage()."]"
."\n\tFILE[".$exception->getFile()."] Line[".$exception->getLine()."]"
."\n\tREQUEST[$sReq] REFERER[$sRef]"
."\n\tAGENT[$sUserAgent] ADDR[$sRemoteAddr]\n"
;
}
catch (Exception $e)
{
$sErrMessage = $exception->getMessage();
}
return $sErrMessage;
}
Customize this method to your needs. Make sure it doesn't raise any exceptions itself.
Now you need 2 more small changes in both sfException.php and sfError404Exception. Change the line:
error_log($this->getMessage());
to:
error_log($this->getMessageFull());
More Enhancements?
It's your call. You can email yourself an alert, include more system-specific pieces of information or use the code as is. It's obviously not the cleanest solution possible but it works for me and hope it helps you.
Wednesday, February 25, 2009
3,500 collectors on Colnect
How do collectors learn about Colnect?
As Colnect allows collectors to manage their personal collection and semi-automatically manage swaps with other collectors from around the world, it's the collectors' own interest to have their collector friends join the site. Why would a collector go through the swap and wish lists of another collector manually when a match to his own lists can be done using a single click?
So what's next?
Colnect will expand to more collectible fields as requests come from existing members of the community. Some requested premium services are also planned.
Thursday, February 19, 2009
BE CAREFUL when using both AdSense + AdWords by Google
Do you have a website and using both AdSense and AdWords? If so, this post will interest you. You may start by looking at the attached picture.
AdWords and AdSense are the different sides of the same coin and Google is in the middle, biting on the metal. AdSense allows website owners to show different ads from different advertisers on their website and receive revenues for it. AdWords allow you to advertise your website on many other websites running AdSense. Google takes its commission for providing both these services. Though many other competing programs exist on the web, Google's generally have good reputation.
Advertising my site on my site?!?!?!
This morning, I was amazed to see an ad to my website for collectors on the same site! Had I clicked this ad, my AdWords account would have been billed for the click while my AdSense account would have received some revenue. Obviously the latter would be lesser since Google get their cut. This is, of course, ridiculous and should be automatically prevented by Google. Apparently, it isn't. Just to ensure you that I'm not an idiot, the ad is for colnect.com and the website is on the same domain - colnect.com
I usually view my own site with ads disabled so I don't accidentally click an ad and violate the terms of service. This time I was at a friend's house and am quite happy to have stumbled upon the aforementioned bug/feature.
Quick Remedy
Both accounts allow you to use filters and I've now added "competition filters" for my AdSense account that will prevent showing ads for any of my domain. It's also possible to filter out sites on AdWords but this can seemingly be done only on the campaign level so if you're running many different campaigns, it'll become tedious. See the attached pictures.
Saturday, February 14, 2009
Upto 80% Speed Increase on Colnect with Symfony
Partials are evil
Maybe not that evil but they take their toll on your loading time. The worst is when using them inside a loop as the price increases linearly. Use helpers whenever possible but make sure you load only the necessary helpers on each call and don't try to re-load helpers when inside a loop.
Re-use function results
Symfony encourages you to use methods and functions repeatedly, for example
sfContext::getInstance()->getModuleName();
. Obviously, the more calculations, the longer things take so whenever you need to re-use results, save the variable content.A good PHP structure for re-using results is:
function foo_calculate() {
static $result = null;
if (is_null($result)) {
# perform calculations
$result = calculation results...
}
return $result;
}
Escaping PHP and going back to PHP takes its toll
Using
?>xxx
is more costly than
echo 'xxx';
See code on the next paragraph.
Using many echo calls slows things down
It's better to accumulate output in a variable and call a single echo.
The following code performs simple tests so you can get a feeling of the differences in execution times. Run it a few times (when all other applications are closed) since results alter a bit every time.
public function executeCompareOutput(sfWebRequest $request) {
$times = 100000;
echo 'Looping for '.$times.' times - results in msec';
echo '<-div style="display:none">';
$start = microtime(true);
for ($x = 0; $x++ < $times;) {
echo ' '.$x;
}
echo '<-/div><-br/>'.round(1000 * (microtime(true) - $start));
$start = microtime(true);
echo '<-div style="display:none">';
for ($x = 0; $x++ < $times;) {
?> echo $x;
}
echo '-div><-br/>'.round(1000 * (microtime(true) - $start));
$start = microtime(true);
echo '<-div style="display:none">';
$sBuf = '';
for ($x = 0; $x++ < $times;) {
$sBuf .= ' '.$x;
}
echo $sBuf;
echo '-div><-br/>'.round(1000 * (microtime(true) - $start));
$start = microtime(true);
echo '<-div style="display:none">';
$sBuf = '';
for ($x = 0; $x++ < $times;) {
$sBuf .= ' ';
$sBuf .= $x;
}
echo $sBuf;
echo '-div><-br/>'.round(1000 * (microtime(true) - $start));
$start = microtime(true);
echo '<-div style="display:none">';
$GLOBALS['bufbuf'] = '';
for ($x = 0; $x++ < $times;) {
$GLOBALS['bufbuf'] .= ' '.$x;
}
echo $sBuf;
echo '-div><-br/>'.round(1000 * (microtime(true) - $start));
die('<-br/>bye');
}
Here is a sample output:
Looping for 10000 times - results in msec
3044
5503
10
15
16
bye
On this run using multiple echo calls + PHP escaping took 55 TIMES MORE than buffering the output in a variable. This clearly proves that the style suggested by Symfony templates, using many PHP echo blocks, is HIGHLY inefficient. If you have a few dozens of it in your templates and your content is cached, this is negligible. If your content is very dynamic, as is the case with Colnect, we're talking about something very worth noting.
Wednesday, February 11, 2009
Colnect's Alexa Ranking Keeps Going Up
To those who may not know, Alexa ranks many (most?) websites out there of the big WWW. It does that by analyzing the traffic of (mostly unsuspecting?) users who install the Alexa toolbar on their browsers. This nice toolbar delivers the information back to their big servers, they crunch it whichever way they like and then rank the sites according to this information.
Why would anyone care about Alexa's ranking? Well, it has come to be relatively important in the WWW world. It allows you to have some (allegedly non-biased) objective information about how your website fairs on the Internet.
Are the results reliable? The short answer IMHO: no. The longer answer: perhaps, to some length. Since some website owners/operators/SEO personnel think of it as important, they are making an effort to get their ranking up. This isn't that hard, considering the fact the grand majority of users don't have the Alexa toolbar. There are many methods but they're basically about making people who use your site have the Alexa toolbar thus your site will have an improved ranking.
Another important disadvantage of Alexa is that ranking is on a per-domain basis. As such, this blog (which a few read) ranks the same as Colnect (which many frequently use). Personal websites on some free hosting all rank the same so you cannot tell them apart and so on.
Does Colnect try to improve its ranking? No. I did think about it for some time, since after all it may look better to some people, but have decided not to. The main two reasons would be not to spend my time on metrics that don't really matter and the second that since I wouldn't install it on my machine, I wouldn't ask others to do so.
Okay, let's hope this post won't cause Colnect's Alexa ranking to crash :)
Sunday, February 8, 2009
Numbers, Numbers, Numbers...
The big benefit becomes clearer when looking at a collector's collection, swap list or wish list. It's then very easy to know how many items the collector has of each country, company or even series.
Here's for example the information for an expert collector on Colnect, Dravec
Phonecards
Collection: 7,546 Phonecards
Swap list: 1,290 Phonecards (Match with my wish list)
Wish list: 6,199 Phonecards (Match with my swap list)
Stamps
Collection: 1,137 Stamps
Swap list: 3 Stamps (Match with my wish list)
Wish list: 5,678 Stamps (Match with my swap list)
Coins
Collection: 2,155 Coins
Swap list: 301 Coins (Match with my wish list)
Wish list: 10,938 Coins (Match with my swap list)
On the back end side of Colnect, the system is very flexible in supplying the given information so the real challenge was to try and create an as-intuitive-as-possible user interface. I've recently made a post about usability and the addition of counters and simplification of the user interface that followed is a big step forward.
Link and Search
Did you like reading it? Stay in the loop via RSS. Thanks :)