Discussion of Wiki Topic Publishing.
The major design issue here is push or pull? If you change a topic in a subscribed wiki, when does it appear in the subscribing wiki?
If it is to be immediate then the design should consist of a series of real-time calls to the subscribed wiki. The impact of immediate update is somewhat mitigated by minimizing the number of wikis involved in a particular operation, but cartying this to the extreme defeats the purpose.
If not immediately then more data can be cached locally, updates can occur less frequently both of which willl improve performance.
So how often is not often enough? 1 Minute, 1 Hour, 1 Day? Should it be configurable by wiki? By user?
Seems like the answer to the push/pull question will depend on a number of dynamic factors including: wiki traffic, message size, benefit of fresh content, cost of stale content, cost of bandwidth etc. Given this, why not adopt a PushOrPull option each message type? Busy, transactional wikis may opt to pull info on their time-line. Quiet, analytical wikis may opt to have updates pushed to it. Some random thoughts follow: - ?lc
Each wiki should be able to control, to some extent the traffic it receives from another. EG: WikiA is being pounded by 1 minute interval upates pushed from WikiB, thus WikiA has the option of switching to pull mode from WikiB, and then pull the updates at a 1 hour interval.
Wemight consider a "Push Events, Pull Content" strategy where small "I've got stuff for you" messages are pushed, allowing the recievers to pull the larger content lists when they are ready for it.
Either way, all of this communication looks like async observer material to me. I don't think we want a situation where a user's edit update response is delayed while the change is pushed to subscribers. Better to have a short-interval observer push those updates and serve the user quickly.
I think from a conceptual point of view "we" should not allow ourselves to be lead by these considerations; in my experience when all is normalized how it really should, you end up with the least data-transfer possible. Very generally spoken : if you appearently end up with too much data-transfer somewhere something is (still) wrong. What I say here is, that the (too much) data-transfer may be used as a measurement to find out whether things were normalized right. Now note that since all of the underlaying topics here (for now) deal about this subject, we all feel that we are not there yet for this matter. I am pretty sure that when we keep on asking questions c.q. put in the open the questioning whether things are right *and* the responses remain as they are uptill now, in the end it 'll be allright. Mind you, we are normalizing processes here which is much tougher than normalizing data. However, when starting at the data (objects if you like or even better : phenomenons) the processes may be normalized the right way automatically. My idea : try to avoid all the technical stuff in the conceptual stage, but don't forget to look at that too. First design at the logical level and iterate immedately to the techniques (available). -- Peter Stordiau
I would say all should be triggered by subscribing and thus not replicating. Seems more logic *and* the problem has gone. Or ? -- Peter Stordiau
I agree that it is more logical to publish all the information as a web service, however this approach is more in line with the goal of level3 and this is a level 2 topic. Also how many different wikis are likely to implement an architecture as complex as level 3?
My thinking is that a simple level2 architecture could be layered onto the current architecture without much difficulty. This would provide many of the benefits of level 3 with a much lower cost and time to market. A major goal of the level 2 architecture is gain access to as many wikis as possible. This is also facilitated by a simple level 2 architecture. ?jMM
So we push; as a negative thinker (always :-)) I come to the next : suppose the subscribing Wiki (pulling) creates a topic (ie potential link to begin with) it should - or anyhow can ask for the extistence of it (to all Wikis ?). But now the pushing (publishing / replicating) Wiki just pushes topics indeed. -> Can't work this out so quikly to wrong stuff, but it seems so wrong that a creating Wiki now can't look for the existence of the topic. Please mind the difference : "Please describe xxxx here" (push situation) vs. "here's your invented link already - oops, define another word" (pull situation). Level2 or 3 (agreed about that) I hope you feel what I mean because it is hard to express. -- Peter Stordiau (switching off lights)
(back again, lights off : No, I think I am wrong; 2 = 2 or ... pfff confused. Wasn't it about push or pull ? Sorry).
The mixing of push and pull looks natural to me, when the pull is looked at as polling. However, this polling doesn't feel good when we see that the number of sites to poll virtally is inifinite. The push of messages "I have new data" is a kind of push-poll which lead to the same : to how many sites ? This may lead to a central messaging server where all the push-polls go to (the data not being there). Now where this looks allright by itself, I wonder whether this may be part of the solution; what it needs (??) is a central web-server facilitated by ... whom ? However, when this is allowed to be the solution, the data-transfers will be minimized;
When a topic is changed, a message about this is put on the central server; this is the push-part.
The central server may be a kinda Wiki too, containing all the info about which messages c.q. underlaying data belongs to which publisher.
Each Wiki polls the central server for data it's subsribed to; once a minute ? no problem since it's only one poll. The pull-part.
When there is data available for a subscriber it connects to the originator and does a download leaving the central server alone.
When 1000's of Wikis take part of this, and each has 100 new/changed messages per 10 hours, this is 10 messages per hour = 0,17 messages per minute. This may lead to 170 messages per minute from all 1,000 Wiki's which is not so much realistic because not all subscribe to all others.
So from the logical (normalized) point of view it is solved with no technical problems, but I don't like the anonymous central server ...
-- Peter Stordiau
The centralized server idea is tempting, but I think it is something we should avoid at all costs. Introducing a single point of control (and failure) will not scale well technically or politically. However, having a transactional wiki server also tasked with dealing with a lot of inter-wiki traffic won’t scale well either. Two thoughts:
This is not a problem that will require immediate solutions. Anticipating likely solutions will do.
If the push (outgoing) communications are observer based, those observers can be running on a different server, potentially using a different internet connection.
Consider the idea of a WikiProxy server which would be dedicated to providing the interface for Wiki-To-Wiki communications for a subset of wikis with a single owner. All incoming transactions are handled by the Proxy, leaving the actual wiki servers free to serve users. - Lauren Clarke
Is there an idea in a WikiSuffix to work out explicit unique topics like -- PeterStordiau@WikiA ?
Category Global Wiki Category Global Wiki Threads
( Topic last updated: 2001.05.21 12:09:30 PM )